mm: migrate: Fix THP's mapcount on isolation
Subsystems (2)
PATCH v2 1/1 mm: migrate: Fix THP's mapcount on isolation
The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.
Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.
---
v2: Corrected fix tag and increase page's refcount before the check
--- mm/compaction.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)
mm/compaction.c +11 -11
@@ -984,29 +984,29 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, goto isolate_fail; } +/* +* Be careful not to clear PageLRU until after we're +* sure the page is not being freed elsewhere -- the +* page release code relies on it. +*/ +if (unlikely (!get_page_unless_zero (page))) + goto isolate_fail; +/* * Migration will fail if an anonymous page is pinned in memory, * so avoid taking lru_lock and isolating it unnecessarily in an * admittedly racy check. */ mapping =page_mapping (page); -if (!mapping &&page_count (page) >page_mapcount (page)) - goto isolate_fail; +if (!mapping && (page_count (page) -1 ) >total_mapcount (page)) + goto isolate_fail_put;/* * Only allow to migrate anonymous pages in GFP_NOFS context * because those do not depend on fs locks. */ if (!(cc->gfp_mask & __GFP_FS) && mapping) - goto isolate_fail; - -/* -* Be careful not to clear PageLRU until after we're -* sure the page is not being freed elsewhere -- the -* page release code relies on it. -*/ -if (unlikely (!get_page_unless_zero (page))) - goto isolate_fail; + goto isolate_fail_put;/* Only take pages on LRU: a check now makes later tests safe */ if (!PageLRU (page))
Signature · 1 line
2.23.0
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.
Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.
Did you look into handling pages that are in the swapcache case as well?
See is_refcount_suitable() in mm/khugepaged.c.
Should be easy to reproduce, let me know if you need inspiration.
Quoted 57 lines
Fixes 1da2f328fa64 ("mm,thp,compaction,cma: allow THP migration for CMA allocations")Cc [email protected] # v5.7+Reported-by Zhenyu Zhang <[email protected]>Suggested-by David Hildenbrand <[email protected]>Signed-off-by Gavin Shan <[email protected]>---
v2: Corrected fix tag and increase page's refcount before the check--- mm/compaction.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)mm/compaction.c +11
-11@@ -984,29 +984,29 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, goto isolate_fail; } +/* +* Be careful not to clear PageLRU until after we're +* sure the page is not being freed elsewhere -- the +* page release code relies on it. +*/ +if (unlikely (!get_page_unless_zero (page))) + goto isolate_fail; +/* * Migration will fail if an anonymous page is pinned in memory, * so avoid taking lru_lock and isolating it unnecessarily in an * admittedly racy check. */ mapping =page_mapping (page); -if (!mapping &&page_count (page) >page_mapcount (page)) - goto isolate_fail; +if (!mapping && (page_count (page) -1 ) >total_mapcount (page)) + goto isolate_fail_put;/* * Only allow to migrate anonymous pages in GFP_NOFS context * because those do not depend on fs locks. */ if (!(cc->gfp_mask & __GFP_FS) && mapping) - goto isolate_fail; - -/* -* Be careful not to clear PageLRU until after we're -* sure the page is not being freed elsewhere -- the -* page release code relies on it. -*/ -if (unlikely (!get_page_unless_zero (page))) - goto isolate_fail; + goto isolate_fail_put;/* Only take pages on LRU: a check now makes later tests safe */ if (!PageLRU (page))
Signature · 3 lines
Thanks, David / dhildenb
Quoted 14 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.
Quoted 6 lines
Did you look into handling pages that are in the swapcache case as well?
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
Quoted 46 lines
Quoted 46 lines
Fixes 1da2f328fa64 ("mm,thp,compaction,cma: allow THP migration for CMA allocations")Cc [email protected] # v5.7+Reported-by Zhenyu Zhang <[email protected]>Suggested-by David Hildenbrand <[email protected]>Signed-off-by Gavin Shan <[email protected]>---
v2: Corrected fix tag and increase page's refcount before the check
---
mm/compaction.c| 22 +++++++++++-----------1 file changed, 11 insertions(+), 11 deletions(-)mm/compaction.c +10
-6@@ -984,29 +984,29 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, goto isolate_fail; } + /* + * Be careful not to clear PageLRU until after we're + * sure the page is not being freed elsewhere -- the + * page release code relies on it. + */ + if (unlikely(!get_page_unless_zero(page))) + goto isolate_fail; + /* * Migration will fail if an anonymous page is pinned in memory, * so avoid taking lru_lock and isolating it unnecessarily in an * admittedly racy check. */ mapping = page_mapping(page); - if (!mapping && page_count(page) > page_mapcount(page)) - goto isolate_fail; + if (!mapping && (page_count(page) - 1) > total_mapcount(page)) + goto isolate_fail_put; /* * Only allow to migrate anonymous pages in GFP_NOFS context * because those do not depend on fs locks. */ if (!(cc->gfp_mask & __GFP_FS) && mapping) - goto isolate_fail; - - /* -* Be careful not to clear PageLRU until after we're
Quoted 9 lines
mm/compaction.c isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, · line 1012
Quoted 8 lines
mm/compaction.c +1
-5-* sure the page is not being freed elsewhere -- the -* page release code relies on it. -*/ - if (unlikely(!get_page_unless_zero(page))) - goto isolate_fail; + goto isolate_fail_put; /* Only take pages on LRU: a check now makes later tests safe */ if (!PageLRU(page))
Thanks,
Gavin
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.Did you look into handling pages that are in the swapcache case as well?
Quoted 7 lines
Quoted 4 lines
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
A simple reproducer would be (on a system with ordinary swap (not zram))
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
2) Enable THP for that region (MADV_HUGEPAGE)
3) Populate a THP (e.g., write access)
4) PTE-map the THP, for example, using MADV_FREE on the last subpage
5) Trigger swapout of the THP, for example, using MADV_PAGEOUT
6) Read-access to some subpages to fault them in from the swapcache
Now you'd have a THP, which
1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)
Now we could test, if alloc_contig_range() will still succeed (e.g., using virtio-mem).
Signature · 3 lines
Thanks, David / dhildenb
Zi Yan
reviewerQuoted 31 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.Did you look into handling pages that are in the swapcache case as well?
Quoted 7 lines
Quoted 4 lines
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
A simple reproducer would be (on a system with ordinary swap (not zram))
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
Quoted 7 lines
2) Enable THP for that region (
MADV_HUGEPAGE)3) Populate a THP (e.g., write access)
4) PTE-map the THP, for example, using
MADV_FREEon the last subpage5) Trigger swapout of the THP, for example, using
MADV_PAGEOUT
Added the original THP swapout code author, Ying.
At this step, the THP will be split, right?
https://elixir.bootlin.com/linux/latest/source/mm/vmscan.c#L1786
Even if a THP has PMD mapping, IIRC, it is split in the add_to_swap() then swapped out. But I cannot find that split code now.
Quoted 4 lines
6) Read-access to some subpages to fault them in from the swapcache
Quoted 12 lines
Now you'd have a THP, which
1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)Now we could test, if
alloc_contig_range()will still succeed (e.g., using virtio-mem).Signature · 3 lines
Thanks, David / dhildenb
--
Best Regards,
Yan Zi
Quoted 42 lines · Zi Yan
Quoted 31 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.Did you look into handling pages that are in the swapcache case as well?
Quoted 7 lines
Quoted 4 lines
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
A simple reproducer would be (on a system with ordinary swap (not zram))
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
Quoted 7 lines
2) Enable THP for that region (
MADV_HUGEPAGE)3) Populate a THP (e.g., write access)
4) PTE-map the THP, for example, using
MADV_FREEon the last subpage5) Trigger swapout of the THP, for example, using
MADV_PAGEOUT
Quoted 8 lines
Added the original THP swapout code author, Ying.
At this step, the THP will be split, right?
https://elixir.bootlin.com/linux/latest/source/mm/vmscan.c#L1786
Even if a THP has PMD mapping, IIRC, it is split in the
add_to_swap()then swapped out. But I cannot find that split code now.
I recall there was some sequence to achieve it. Maybe it was swapping out the PMD first and not triggering a PTE-mapping first.
mm/vmscan.c:shrink_folio_list()
if (folio_test_large(folio)) {
/* cannot split folio, skip it */
if (!can_split_folio(folio, NULL))
goto activate_locked;
/*
* Split folios without a PMD map right
* away. Chances are some or all of the
* tail pages can be freed without IO.
*/
if (!folio_entire_mapcount(folio) &&
split_folio_to_list(folio, folio_list))
goto activate_locked;
}
}So the sequence might have to be
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
2) Enable THP for that region (MADV_HUGEPAGE)
3) Populate a THP (e.g., write access)
4) Trigger swapout of the THP, for example, using MADV_PAGEOUT
5) Access some subpage
As we don't have PMD swap entries, we will PTE-map the THP during try_to_unmap() IIRC.
Independent of that, the check we have here also doesn't consider ordinary order-0 pages that might be in the swapache.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 42 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.Did you look into handling pages that are in the swapcache case as well?
Quoted 7 lines
Quoted 4 lines
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
A simple reproducer would be (on a system with ordinary swap (not zram))
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
2) Enable THP for that region (
MADV_HUGEPAGE)3) Populate a THP (e.g., write access)
4) PTE-map the THP, for example, using
MADV_FREEon the last subpage5) Trigger swapout of the THP, for example, using
MADV_PAGEOUT6) Read-access to some subpages to fault them in from the swapcache
Quoted 8 lines
Now you'd have a THP, which
1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)Now we could test, if
alloc_contig_range()will still succeed (e.g., using virtio-mem).
Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.
I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.
# numactl -p 1 testsuite mm swap -k
Any key to split THP
Any key to swap sub-pages
Any key to read the swapped sub-pages
Page[000]: 0xffffffffffffffff
Page[001]: 0xffffffffffffffff
:
Page[255]: 0xffffffffffffffff
Any key to exit // hold here and the program doesn't exit(qemu) qom-set vm1 requested-size 0
[ 356.005396] virtio_mem virtio1: plugged size: 0x40000000 [ 356.005996] virtio_mem virtio1: requested size: 0x0 [ 356.350299] Fallback order for Node 0: 0 1 [ 356.350810] Fallback order for Node 1: 1 0 [ 356.351260] Built 2 zonelists, mobility grouping on. Total pages: 491343 [ 356.351998] Policy zone: DMA
Thanks,
Gavin
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <errno.h>
#include "inc/testsuite.h"
struct swap_struct {
int page_size;
int fd;
int flags;
void *buf;
unsigned long len;
int key_break;
};#define SWAP_DEFAULT_SIZE 0x200000 /* 2MB */
#define SWAP_PAGE_TO_SPLIT 511
#define SWAP_PAGE_TO_SWAP 1
#define SWAP_PAGE_TO_SWAP_NUM 256
#ifndef MADV_PAGEOUT
#define MADV_PAGEOUT 21
#endif
static void usage(void)
{
fprintf(stdout, "testsuite mm swap -l <size> -k\n");
fprintf(stdout, "\n");
fprintf(stdout, "-l: Length of memory to be mapped\n");
fprintf(stdout, "-w: Length of memory to be copied-on-write\n");
fprintf(stdout, "-k: Stop at various stages\n");
fprintf(stdout, "\n");
}static int swap_init_data(struct swap_struct *m)
{
m->page_size = getpagesize();
m->fd = -1;
m->flags = (MAP_PRIVATE | MAP_ANONYMOUS);
m->len = SWAP_DEFAULT_SIZE;
m->key_break = 0;
return 0;
}static int swap_handler(int argc, char **argv)
{
struct swap_struct m;
unsigned long *pval;
int i, opt, ret;
ret = swap_init_data(&m);
if (ret)
return ret;
while ((opt = getopt(argc, argv, "l:w:kh")) != -1) {
switch (opt) {
case 'l':
m.len = util_memory_parse_size(optarg);
if (m.len <= SWAP_DEFAULT_SIZE) {
fprintf(stderr, "%s: length 0x%lx less than 0x%x\n",
__func__, m.len, SWAP_DEFAULT_SIZE);
return -1;
}
break;
case 'k':
m.key_break = 1;
break;
case 'h':
usage();
return 0;
}
}
/*
* Setup the area. The area should be backed up with huge pages
* if it suits. Write to the area to ensure the area is populted
* completely.
*/
m.buf = mmap(NULL, m.len, PROT_READ | PROT_WRITE, m.flags, m.fd, 0);
if (m.buf == (void *)-1) {
fprintf(stderr, "Unable do mmap()\n");
goto out;
}
memset(m.buf, 0xff, m.len);
/* Force to split the huge page */
util_misc_key_press(m.key_break, " ", "Any key to split THP");
ret = madvise(m.buf + SWAP_PAGE_TO_SPLIT * m.page_size,
m.page_size, MADV_FREE);
if (ret) {
fprintf(stderr, "Error %d to split THP\n", ret);
goto out;
}
/* Swap one sub-page */
util_misc_key_press(m.key_break, " ", "Any key to swap sub-pages");
ret = madvise(m.buf + SWAP_PAGE_TO_SWAP * m.page_size,
SWAP_PAGE_TO_SWAP_NUM * m.page_size,
MADV_PAGEOUT);
if (ret) {
fprintf(stderr, "Error %d to swap one sub-page\n", ret);
goto out;
}
/* Read the swapped sub-page */
util_misc_key_press(m.key_break, " ", "Any key to read the swapped sub-pages");
for (i = 0; i < SWAP_PAGE_TO_SWAP_NUM; i++) {
pval = (unsigned long *)(m.buf + (SWAP_PAGE_TO_SWAP + i) * m.page_size);
fprintf(stdout, " Page[%03d]: 0x%016lx\n", i, *pval);
}
/* Exit */
util_misc_key_press(m.key_break, " ", "Any key to exit");out:
if (m.buf != (void *)-1) munmap(m.buf, m.len); if (m.fd > 0) close(m.fd); return 0; }
static struct command swap_command = {
.name = "swap",
.handler = swap_handler,
.children = LIST_HEAD_INIT(swap_command.children),
.link = LIST_HEAD_INIT(swap_command.link),
};int mm_swap_init(void)
{
return command_add(&mm_command, &swap_command);
}Quoted 49 lines · Gavin Shan
Quoted 48 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.Did you look into handling pages that are in the swapcache case as well?
Quoted 7 lines
Quoted 4 lines
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
A simple reproducer would be (on a system with ordinary swap (not zram))
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
2) Enable THP for that region (
MADV_HUGEPAGE)3) Populate a THP (e.g., write access)
4) PTE-map the THP, for example, using
MADV_FREEon the last subpage5) Trigger swapout of the THP, for example, using
MADV_PAGEOUT6) Read-access to some subpages to fault them in from the swapcache
Now you'd have a THP, which
1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)
Quoted 8 lines
Quoted 2 lines
Now we could test, if
alloc_contig_range()will still succeed (e.g., using virtio-mem).Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.
I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.
Interesting. But I don't really see how we could pass this check with a page that's in the swapcache, maybe I'm missing something else.
I'll try to see if I can reproduce it.
Signature · 3 lines
Thanks, David / dhildenb
David Hildenbrand
Acked-byQuoted 56 lines · David Hildenbrand
Quoted 55 lines · Gavin Shan
Quoted 42 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.Did you look into handling pages that are in the swapcache case as well?
Quoted 7 lines
Quoted 4 lines
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
A simple reproducer would be (on a system with ordinary swap (not zram))
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
2) Enable THP for that region (
MADV_HUGEPAGE)3) Populate a THP (e.g., write access)
4) PTE-map the THP, for example, using
MADV_FREEon the last subpage5) Trigger swapout of the THP, for example, using
MADV_PAGEOUT6) Read-access to some subpages to fault them in from the swapcache
Quoted 8 lines
Now you'd have a THP, which
1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)Now we could test, if
alloc_contig_range()will still succeed (e.g., using virtio-mem).Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.
Quoted 8 lines
Quoted 2 lines
I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.
Interesting. But I don't really see how we could pass this check with a page that's in the swapcache, maybe I'm missing something else.
I'll try to see if I can reproduce it.
After some unsuccessful attempts and many head-scratches, I realized that it's quite simple why we don't have to worry about swapcache pages here:
page_mapping() is != NULL for pages in the swapcache: folio_mapping() makes this rather obvious:
if (unlikely(folio_test_swapcache(folio))
return swap_address_space(folio_swap_entry(folio));
I think the get_page_unless_zero() might also be a fix for the page_mapping() call, smells like something could blow up on concurrent page freeing. (what about concurrent removal from the swapcache? nobody knows :) )
Thanks Gavin!
Signature · 3 lines
Thanks, David / dhildenb
Zhenyu Zhang
Tested-byWith the patch applied, I'm unable to hit memory hot-remove failure in the environment where the issue was initially found.
Quoted 94 lines · David Hildenbrand
Quoted 56 lines · David Hildenbrand
Quoted 55 lines · Gavin Shan
Quoted 42 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan
The issue is reported when removing memory through
virtio_memdevice. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated inisolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.Quoted 5 lines
Fix it by replacing
page_mapcount()withtotal_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.Did you look into handling pages that are in the swapcache case as well?
Quoted 7 lines
Quoted 4 lines
See
is_refcount_suitable()inmm/khugepaged.c.Should be easy to reproduce, let me know if you need inspiration.
Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.
A simple reproducer would be (on a system with ordinary swap (not zram))
1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP
2) Enable THP for that region (
MADV_HUGEPAGE)3) Populate a THP (e.g., write access)
4) PTE-map the THP, for example, using
MADV_FREEon the last subpage5) Trigger swapout of the THP, for example, using
MADV_PAGEOUT6) Read-access to some subpages to fault them in from the swapcache
Quoted 8 lines
Now you'd have a THP, which
1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)Now we could test, if
alloc_contig_range()will still succeed (e.g., using virtio-mem).Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.
Quoted 8 lines
Quoted 2 lines
I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.
Interesting. But I don't really see how we could pass this check with a page that's in the swapcache, maybe I'm missing something else.
I'll try to see if I can reproduce it.
After some unsuccessful attempts and many head-scratches, I realized that it's quite simple why we don't have to worry about swapcache pages here:
page_mapping()is != NULL for pages in the swapcache:folio_mapping()makes this rather obvious:if (unlikely(folio_test_swapcache(folio))
return swap_address_space(folio_swap_entry(folio));I think the
get_page_unless_zero()might also be a fix for thepage_mapping()call, smells like something could blow up on concurrent page freeing. (what about concurrent removal from the swapcache? nobody knows :) )Thanks Gavin!
--
Thanks,David / dhildenb