Skip to contentSkip to conversation

mm: migrate: Fix THP's mapcount on isolation

PATCH Applied Gavin Shan 4 participants 10 messages Nov 24, 2022 – Nov 25, 2022

Subsystems (2)
lore
Share & tools
lore

Gavin Shan

author
#0 lore raw

PATCH v2 1/1 mm: migrate: Fix THP's mapcount on isolation

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Fixes 1da2f328fa64 ("mm,thp,compaction,cma: allow THP migration for CMA allocations")

---
v2: Corrected fix tag and increase page's refcount before the check

---
 mm/compaction.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
mm/compaction.c +11 -11
index c51f7f545afe..1f6da31dd9a5 100644
@@ -984,29 +984,29 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 			goto isolate_fail;
 		}
 
+		/*
+		 * Be careful not to clear PageLRU until after we're
+		 * sure the page is not being freed elsewhere -- the
+		 * page release code relies on it.
+		 */
+		if (unlikely(!get_page_unless_zero(page)))
+			goto isolate_fail;
+
 		/*
 		 * Migration will fail if an anonymous page is pinned in memory,
 		 * so avoid taking lru_lock and isolating it unnecessarily in an
 		 * admittedly racy check.
 		 */
 		mapping = page_mapping(page);
-		if (!mapping && page_count(page) > page_mapcount(page))
-			goto isolate_fail;
+		if (!mapping && (page_count(page) - 1) > total_mapcount(page))
+			goto isolate_fail_put;
 
 		/*
 		 * Only allow to migrate anonymous pages in GFP_NOFS context
 		 * because those do not depend on fs locks.
 		 */
 		if (!(cc->gfp_mask & __GFP_FS) && mapping)
-			goto isolate_fail;
-
-		/*
-		 * Be careful not to clear PageLRU until after we're
-		 * sure the page is not being freed elsewhere -- the
-		 * page release code relies on it.
-		 */
-		if (unlikely(!get_page_unless_zero(page)))
-			goto isolate_fail;
+			goto isolate_fail_put;
 
 		/* Only take pages on LRU: a check now makes later tests safe */
 		if (!PageLRU(page))
Signature · 1 line
2.23.0
#1 lore raw
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Quoted 57 lines
Fixes 1da2f328fa64 ("mm,thp,compaction,cma: allow THP migration for CMA allocations")

---
v2: Corrected fix tag and increase page's refcount before the check

---
  mm/compaction.c | 22 +++++++++++-----------
  1 file changed, 11 insertions(+), 11 deletions(-)
mm/compaction.c +11 -11
index c51f7f545afe..1f6da31dd9a5 100644
@@ -984,29 +984,29 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
  			goto isolate_fail;
  		}
 
+		/*
+		 * Be careful not to clear PageLRU until after we're
+		 * sure the page is not being freed elsewhere -- the
+		 * page release code relies on it.
+		 */
+		if (unlikely(!get_page_unless_zero(page)))
+			goto isolate_fail;
+
  		/*
  		 * Migration will fail if an anonymous page is pinned in memory,
  		 * so avoid taking lru_lock and isolating it unnecessarily in an
  		 * admittedly racy check.
  		 */
  		mapping = page_mapping(page);
-		if (!mapping && page_count(page) > page_mapcount(page))
-			goto isolate_fail;
+		if (!mapping && (page_count(page) - 1) > total_mapcount(page))
+			goto isolate_fail_put;
 
  		/*
  		 * Only allow to migrate anonymous pages in GFP_NOFS context
  		 * because those do not depend on fs locks.
  		 */
  		if (!(cc->gfp_mask & __GFP_FS) && mapping)
-			goto isolate_fail;
-
-		/*
-		 * Be careful not to clear PageLRU until after we're
-		 * sure the page is not being freed elsewhere -- the
-		 * page release code relies on it.
-		 */
-		if (unlikely(!get_page_unless_zero(page)))
-			goto isolate_fail;
+			goto isolate_fail_put;
 
  		/* Only take pages on LRU: a check now makes later tests safe */
  		if (!PageLRU(page))
Signature · 3 lines
Thanks,

David / dhildenb
author
#2 lore raw
Quoted 14 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Quoted 6 lines

Did you look into handling pages that are in the swapcache case as well?

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

Quoted 46 lines
Quoted 46 lines
Fixes 1da2f328fa64 ("mm,thp,compaction,cma: allow THP migration for CMA allocations")
Cc [email protected]   # v5.7+

---
v2: Corrected fix tag and increase page's refcount before the check
---
  mm/compaction.c | 22 +++++++++++-----------

  1 file changed, 11 insertions(+), 11 deletions(-)
mm/compaction.c +10 -6
index c51f7f545afe..1f6da31dd9a5 100644
@@ -984,29 +984,29 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
               goto isolate_fail;
           }
+        /*
+         * Be careful not to clear PageLRU until after we're
+         * sure the page is not being freed elsewhere -- the
+         * page release code relies on it.
+         */
+        if (unlikely(!get_page_unless_zero(page)))
+            goto isolate_fail;
+
           /*
            * Migration will fail if an anonymous page is pinned in memory,
            * so avoid taking lru_lock and isolating it unnecessarily in an
            * admittedly racy check.
            */
           mapping = page_mapping(page);
-        if (!mapping && page_count(page) > page_mapcount(page))
-            goto isolate_fail;
+        if (!mapping && (page_count(page) - 1) > total_mapcount(page))
+            goto isolate_fail_put;
           /*
            * Only allow to migrate anonymous pages in GFP_NOFS context
            * because those do not depend on fs locks.
            */
           if (!(cc->gfp_mask & __GFP_FS) && mapping)
-            goto isolate_fail;
-
-        /*
-         * Be careful not to clear PageLRU until after we're
Quoted 9 lines

mm/compaction.c isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, · line 1012

Quoted 8 lines
mm/compaction.c +1 -5
-         * sure the page is not being freed elsewhere -- the
-         * page release code relies on it.
-         */
-        if (unlikely(!get_page_unless_zero(page)))
-            goto isolate_fail;
+            goto isolate_fail_put;
           /* Only take pages on LRU: a check now makes later tests safe */
           if (!PageLRU(page))

Thanks,
Gavin

#3 lore raw
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

Quoted 7 lines
Quoted 4 lines

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

A simple reproducer would be (on a system with ordinary swap (not zram))

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) PTE-map the THP, for example, using MADV_FREE on the last subpage

5) Trigger swapout of the THP, for example, using MADV_PAGEOUT

6) Read-access to some subpages to fault them in from the swapcache

Now you'd have a THP, which

1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)

Now we could test, if alloc_contig_range() will still succeed (e.g., using virtio-mem).

Signature · 3 lines
Thanks,

David / dhildenb
reviewer
#4 lore raw
Quoted 31 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

Quoted 7 lines
Quoted 4 lines

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

A simple reproducer would be (on a system with ordinary swap (not zram))

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

Quoted 7 lines

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) PTE-map the THP, for example, using MADV_FREE on the last subpage

5) Trigger swapout of the THP, for example, using MADV_PAGEOUT

Added the original THP swapout code author, Ying.

At this step, the THP will be split, right?

https://elixir.bootlin.com/linux/latest/source/mm/vmscan.c#L1786

Even if a THP has PMD mapping, IIRC, it is split in the add_to_swap() then swapped out. But I cannot find that split code now.

Quoted 4 lines

6) Read-access to some subpages to fault them in from the swapcache

Quoted 12 lines

Now you'd have a THP, which

1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)

Now we could test, if alloc_contig_range() will still succeed (e.g., using virtio-mem).

Signature · 3 lines
Thanks,

David / dhildenb

--
Best Regards,
Yan Zi

#5 lore raw
Quoted 42 lines · Zi Yan
Quoted 31 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

Quoted 7 lines
Quoted 4 lines

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

A simple reproducer would be (on a system with ordinary swap (not zram))

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

Quoted 7 lines

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) PTE-map the THP, for example, using MADV_FREE on the last subpage

5) Trigger swapout of the THP, for example, using MADV_PAGEOUT

Quoted 8 lines

Added the original THP swapout code author, Ying.

At this step, the THP will be split, right?

https://elixir.bootlin.com/linux/latest/source/mm/vmscan.c#L1786

Even if a THP has PMD mapping, IIRC, it is split in the add_to_swap() then swapped out. But I cannot find that split code now.

I recall there was some sequence to achieve it. Maybe it was swapping out the PMD first and not triggering a PTE-mapping first.

mm/vmscan.c:shrink_folio_list()

if (folio_test_large(folio)) {
	/* cannot split folio, skip it */
	if (!can_split_folio(folio, NULL))
		goto activate_locked;
	/*
	 * Split folios without a PMD map right
	 * away. Chances are some or all of the
	 * tail pages can be freed without IO.
	 */
	if (!folio_entire_mapcount(folio) &&
	    split_folio_to_list(folio, folio_list))
		goto activate_locked;
	}
}

So the sequence might have to be

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) Trigger swapout of the THP, for example, using MADV_PAGEOUT

5) Access some subpage

As we don't have PMD swap entries, we will PTE-map the THP during try_to_unmap() IIRC.

Independent of that, the check we have here also doesn't consider ordinary order-0 pages that might be in the swapache.

Signature · 3 lines
Thanks,

David / dhildenb
author
#6 lore raw
In reply to David Hildenbrand
Quoted 42 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

Quoted 7 lines
Quoted 4 lines

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

A simple reproducer would be (on a system with ordinary swap (not zram))

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) PTE-map the THP, for example, using MADV_FREE on the last subpage

5) Trigger swapout of the THP, for example, using MADV_PAGEOUT

6) Read-access to some subpages to fault them in from the swapcache

Quoted 8 lines

Now you'd have a THP, which

1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)

Now we could test, if alloc_contig_range() will still succeed (e.g., using virtio-mem).

Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.

I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.

   # numactl -p 1 testsuite mm swap -k
   Any key to split THP
   Any key to swap sub-pages
   Any key to read the swapped sub-pages
       Page[000]: 0xffffffffffffffff
       Page[001]: 0xffffffffffffffff
         :
       Page[255]: 0xffffffffffffffff
   Any key to exit                                // hold here and the program doesn't exit

(qemu) qom-set vm1 requested-size 0

   [  356.005396] virtio_mem virtio1: plugged size: 0x40000000
   [  356.005996] virtio_mem virtio1: requested size: 0x0
   [  356.350299] Fallback order for Node 0: 0 1
   [  356.350810] Fallback order for Node 1: 1 0
   [  356.351260] Built 2 zonelists, mobility grouping on.  Total pages: 491343
   [  356.351998] Policy zone: DMA

Thanks,
Gavin

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <errno.h>

#include "inc/testsuite.h"

struct swap_struct {
	int		page_size;

	int		fd;
	int		flags;
	void		*buf;
	unsigned long 	len;

	int		key_break;
};

#define SWAP_DEFAULT_SIZE 0x200000 /* 2MB */
#define SWAP_PAGE_TO_SPLIT 511
#define SWAP_PAGE_TO_SWAP 1
#define SWAP_PAGE_TO_SWAP_NUM 256

#ifndef MADV_PAGEOUT
#define MADV_PAGEOUT 21
#endif

static void usage(void)

{
	fprintf(stdout, "testsuite mm swap -l <size> -k\n");
	fprintf(stdout, "\n");
	fprintf(stdout, "-l: Length of memory to be mapped\n");
	fprintf(stdout, "-w: Length of memory to be copied-on-write\n");
	fprintf(stdout, "-k: Stop at various stages\n");
	fprintf(stdout, "\n");
}

static int swap_init_data(struct swap_struct *m)

{
	m->page_size	= getpagesize();

	m->fd		= -1;
	m->flags	= (MAP_PRIVATE | MAP_ANONYMOUS);
	m->len		= SWAP_DEFAULT_SIZE;
	m->key_break	= 0;

	return 0;
}

static int swap_handler(int argc, char **argv)

{
	struct swap_struct m;
	unsigned long *pval;
	int i, opt, ret;

	ret = swap_init_data(&m);
	if (ret)
		return ret;

	while ((opt = getopt(argc, argv, "l:w:kh")) != -1) {
		switch (opt) {
		case 'l':
			m.len = util_memory_parse_size(optarg);
			if (m.len <= SWAP_DEFAULT_SIZE) {
				fprintf(stderr, "%s: length 0x%lx less than 0x%x\n",
					__func__, m.len, SWAP_DEFAULT_SIZE);
				return -1;
			}

			break;
		case 'k':
			m.key_break = 1;
			break;
		case 'h':
			usage();
			return 0;
		}
	}

	/*
	 * Setup the area. The area should be backed up with huge pages
	 * if it suits. Write to the area to ensure the area is populted
	 * completely.
	 */
	m.buf = mmap(NULL, m.len, PROT_READ | PROT_WRITE, m.flags, m.fd, 0);
	if (m.buf == (void *)-1) {
		fprintf(stderr, "Unable do mmap()\n");
		goto out;
	}

        memset(m.buf, 0xff, m.len);

	/* Force to split the huge page */
	util_misc_key_press(m.key_break, "  ", "Any key to split THP");
	ret = madvise(m.buf + SWAP_PAGE_TO_SPLIT * m.page_size,
		      m.page_size, MADV_FREE);
	if (ret) {
		fprintf(stderr, "Error %d to split THP\n", ret); 
		goto out;
	}

	/* Swap one sub-page */
	util_misc_key_press(m.key_break, "  ", "Any key to swap sub-pages");
	ret = madvise(m.buf + SWAP_PAGE_TO_SWAP * m.page_size,
		      SWAP_PAGE_TO_SWAP_NUM * m.page_size,
		      MADV_PAGEOUT);
	if (ret) {
		fprintf(stderr, "Error %d to swap one sub-page\n", ret);
		goto out;
	}

	/* Read the swapped sub-page */
	util_misc_key_press(m.key_break, "  ", "Any key to read the swapped sub-pages");
	for (i = 0; i < SWAP_PAGE_TO_SWAP_NUM; i++) {
		pval = (unsigned long *)(m.buf + (SWAP_PAGE_TO_SWAP + i) * m.page_size);
		fprintf(stdout, "  Page[%03d]: 0x%016lx\n", i, *pval);
	}

	/* Exit */
	util_misc_key_press(m.key_break, "  ", "Any key to exit");

out:

	if (m.buf != (void *)-1)
		munmap(m.buf, m.len);
	if (m.fd > 0)
		close(m.fd);

	return 0;
}
static struct command swap_command = {
	.name		= "swap",
	.handler	= swap_handler,
	.children	= LIST_HEAD_INIT(swap_command.children),
	.link		= LIST_HEAD_INIT(swap_command.link),
};

int mm_swap_init(void)

{
	return command_add(&mm_command, &swap_command);
}
#7 lore raw
Quoted 49 lines · Gavin Shan
Quoted 48 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

Quoted 7 lines
Quoted 4 lines

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

A simple reproducer would be (on a system with ordinary swap (not zram))

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) PTE-map the THP, for example, using MADV_FREE on the last subpage

5) Trigger swapout of the THP, for example, using MADV_PAGEOUT

6) Read-access to some subpages to fault them in from the swapcache

Now you'd have a THP, which

1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)

Quoted 8 lines
Quoted 2 lines

Now we could test, if alloc_contig_range() will still succeed (e.g., using virtio-mem).

Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.

I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.

Interesting. But I don't really see how we could pass this check with a page that's in the swapcache, maybe I'm missing something else.

I'll try to see if I can reproduce it.

Signature · 3 lines
Thanks,

David / dhildenb
#8 lore raw
Quoted 56 lines · David Hildenbrand
Quoted 55 lines · Gavin Shan
Quoted 42 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

Quoted 7 lines
Quoted 4 lines

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

A simple reproducer would be (on a system with ordinary swap (not zram))

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) PTE-map the THP, for example, using MADV_FREE on the last subpage

5) Trigger swapout of the THP, for example, using MADV_PAGEOUT

6) Read-access to some subpages to fault them in from the swapcache

Quoted 8 lines

Now you'd have a THP, which

1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)

Now we could test, if alloc_contig_range() will still succeed (e.g., using virtio-mem).

Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.

Quoted 8 lines
Quoted 2 lines

I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.

Interesting. But I don't really see how we could pass this check with a page that's in the swapcache, maybe I'm missing something else.

I'll try to see if I can reproduce it.

After some unsuccessful attempts and many head-scratches, I realized that it's quite simple why we don't have to worry about swapcache pages here:

page_mapping() is != NULL for pages in the swapcache: folio_mapping() makes this rather obvious:

if (unlikely(folio_test_swapcache(folio))

	return swap_address_space(folio_swap_entry(folio));

I think the get_page_unless_zero() might also be a fix for the page_mapping() call, smells like something could blow up on concurrent page freeing. (what about concurrent removal from the swapcache? nobody knows :) )

Thanks Gavin!

Signature · 3 lines
Thanks,

David / dhildenb
Tested-by
#9 lore raw

With the patch applied, I'm unable to hit memory hot-remove failure in the environment where the issue was initially found.

Quoted 94 lines · David Hildenbrand
Quoted 56 lines · David Hildenbrand
Quoted 55 lines · Gavin Shan
Quoted 42 lines · David Hildenbrand
Quoted 17 lines · Gavin Shan
Quoted 16 lines · David Hildenbrand
Quoted 7 lines · Gavin Shan

The issue is reported when removing memory through virtio_mem device. The transparent huge page, experienced copy-on-write fault, is wrongly regarded as pinned. The transparent huge page is escaped from being isolated in isolate_migratepages_block(). The transparent huge page can't be migrated and the corresponding memory block can't be put into offline state.

Quoted 5 lines

Fix it by replacing page_mapcount() with total_mapcount(). With this, the transparent huge page can be isolated and migrated, and the memory block can be put into offline state. Besides, The page's refcount is increased a bit earlier to avoid the page is released when the check is executed.

Did you look into handling pages that are in the swapcache case as well?

Quoted 7 lines
Quoted 4 lines

See is_refcount_suitable() in mm/khugepaged.c.

Should be easy to reproduce, let me know if you need inspiration.

Nope, I didn't look into the case. Please elaborate the details so that I can reproduce it firstly.

A simple reproducer would be (on a system with ordinary swap (not zram))

1) mmap a region (MAP_ANON|MAP_PRIVATE) that can hold a THP

2) Enable THP for that region (MADV_HUGEPAGE)

3) Populate a THP (e.g., write access)

4) PTE-map the THP, for example, using MADV_FREE on the last subpage

5) Trigger swapout of the THP, for example, using MADV_PAGEOUT

6) Read-access to some subpages to fault them in from the swapcache

Quoted 8 lines

Now you'd have a THP, which

1) Is partially PTE-mapped into the page table
2) Is in the swapcache (each subpage should have one reference from the swapache)

Now we could test, if alloc_contig_range() will still succeed (e.g., using virtio-mem).

Thanks for the details. Step (4) and (5) can be actually combined. To swap part of the THP (e.g. one sub-page) will force the THP to be split.

Quoted 8 lines
Quoted 2 lines

I followed your steps in the attached program, there is no issue to do memory hot-remove through virtio-mem with or without this patch.

Interesting. But I don't really see how we could pass this check with a page that's in the swapcache, maybe I'm missing something else.

I'll try to see if I can reproduce it.

After some unsuccessful attempts and many head-scratches, I realized that it's quite simple why we don't have to worry about swapcache pages here:

page_mapping() is != NULL for pages in the swapcache: folio_mapping() makes this rather obvious:

if (unlikely(folio_test_swapcache(folio))

        return swap_address_space(folio_swap_entry(folio));

I think the get_page_unless_zero() might also be a fix for the page_mapping() call, smells like something could blow up on concurrent page freeing. (what about concurrent removal from the swapcache? nobody knows :) )

Thanks Gavin!

--
Thanks,

David / dhildenb