Skip to contentSkip to conversation

mm/compaction: honour compact_unevictable_allowed in mlock race and alloc_contig path

RFC PATCH In review Wandun Chen 5 participants 21 messages Jun 4 – Jul 8

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover mm/compaction: honour compact_unevictable_allowed in mlock race and alloc_contig path

From: Wandun Chen <[email protected]>

vm.compact_unevictable_allowed=0 is meant to keep compaction from touching unevictable folios. In practice there are still two paths where it does not take effect. This series fixes them and adds a tracepoint to make such issues easier to diagnose in the future.

Wandun Chen (3):
mm/compaction: skip isolate mlocked folios when
compact_unevictable_allowed=0
mm/compaction: add per-folio isolation tracepoint mm/compaction: respect compact_unevictable_allowed in alloc_contig
path

 include/linux/compaction.h        |  6 ++++++
 include/trace/events/compaction.h | 26 ++++++++++++++++++++++++++
 mm/compaction.c                   | 14 +++++++++++---
 mm/internal.h                     |  1 +
 mm/page_alloc.c                   |  2 ++
 5 files changed, 46 insertions(+), 3 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

RFC PATCH 1/3 mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

From: Wandun Chen <[email protected]>

compact_unevictable_allowed is default 0 under PREEMPT_RT, isolate_migratepages_block() skips folios with PG_unevictable set. However, mlock_folio() sets PG_mlocked immediately but defers PG_unevictable to mlock_folio_batch(), result in a folio with PG_mlocked=1 but PG_unevictable=0. Compaction will isolate such a folio.

Fix by checking folio_test_mlocked() together with the existing folio_test_unevictable() check.

A similar issue has been reported by Alexander Krabler on a 6.12-rt aarch64 system. Vlastimil suggested to check the mlocked flag [1].

---
 mm/compaction.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/compaction.c +2 -1
index b776f35ad020..7e07b792bcb5 100644
@@ -1116,7 +1116,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		is_unevictable = folio_test_unevictable(folio);
 
 		/* Compaction might skip unevictable pages but CMA takes them */
-		if (!(mode & ISOLATE_UNEVICTABLE) && is_unevictable)
+		if (!(mode & ISOLATE_UNEVICTABLE) &&
+		    (is_unevictable || folio_test_mlocked(folio)))
 			goto isolate_fail_put;
 
 		/*
Signature · 1 line
2.43.0
#2 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 12 lines · Wandun Chen

From: Wandun Chen <[email protected]>

compact_unevictable_allowed is default 0 under PREEMPT_RT, isolate_migratepages_block() skips folios with PG_unevictable set. However, mlock_folio() sets PG_mlocked immediately but defers PG_unevictable to mlock_folio_batch(), result in a folio with PG_mlocked=1 but PG_unevictable=0. Compaction will isolate such a folio.

Fix by checking folio_test_mlocked() together with the existing folio_test_unevictable() check.

Quoted 8 lines

A similar issue has been reported by Alexander Krabler on a 6.12-rt aarch64 system. Vlastimil suggested to check the mlocked flag [1].

Well in that thread, Hugh doubted my suggestion and then it seems we didn't concluded anything. Did you actually in practice observe the issue that Alexander had, and that this patch fixed it, or is that theoretical?

Quoted 18 lines
---
 mm/compaction.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/compaction.c +2 -1
index b776f35ad020..7e07b792bcb5 100644
@@ -1116,7 +1116,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		is_unevictable = folio_test_unevictable(folio);
 
 		/* Compaction might skip unevictable pages but CMA takes them */
-		if (!(mode & ISOLATE_UNEVICTABLE) && is_unevictable)
+		if (!(mode & ISOLATE_UNEVICTABLE) &&
+		    (is_unevictable || folio_test_mlocked(folio)))
 			goto isolate_fail_put;
 
 		/*
author
#3 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 22 lines · Vlastimil Babka (SUSE)
Quoted 12 lines · Wandun Chen

From: Wandun Chen <[email protected]>

compact_unevictable_allowed is default 0 under PREEMPT_RT, isolate_migratepages_block() skips folios with PG_unevictable set. However, mlock_folio() sets PG_mlocked immediately but defers PG_unevictable to mlock_folio_batch(), result in a folio with PG_mlocked=1 but PG_unevictable=0. Compaction will isolate such a folio.

Fix by checking folio_test_mlocked() together with the existing folio_test_unevictable() check.

Quoted 8 lines

A similar issue has been reported by Alexander Krabler on a 6.12-rt aarch64 system. Vlastimil suggested to check the mlocked flag [1].

Quoted 4 lines

Well in that thread, Hugh doubted my suggestion and then it seems we didn't concluded anything. Did you actually in practice observe the issue that Alexander had, and that this patch fixed it, or is that theoretical?

Yes, I wrote a test case that can reproduce it in a few second.

The test case contains 3 steps:
1. mlockall
2. mmap file(2GB) + trigger file write page fault;
3. during step 1, trigger compact via /proc/sys/vm/compact_memory

My reproduction environment is qemu with 4GB ram, 8 core, aarch64, preempt_rt and includes the tracepoint in patch 02. After running the reproduction program for a few seconds, the following output appears.

repro-403     [004] ....1   101.270505: mm_compaction_isolate_folio: pfn=0x71e3a mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270507: mm_compaction_isolate_folio: pfn=0x71e3b mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270513: mm_compaction_isolate_folio: pfn=0x71e3c mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270515: mm_compaction_isolate_folio: pfn=0x71e3d mode=0x0 flags=uptodate|mlocked
repro-403     [004] ....1   101.270517: mm_compaction_isolate_folio: pfn=0x71e3e mode=0x0 flags=uptodate|mlocked
repro-403     [004] ....1   101.270520: mm_compaction_isolate_folio: pfn=0x71e3f mode=0x0 flags=uptodate|mlocked

Unfortunately, I recently found that there is still a bug in the fix patch. Setting mlocked in the mlock_folio function could happen even after the page is successfully isolated, so it still cannot prevent migration. Because of this, I need to think more about how to fix it.

Perhaps we should double-check whether the page is mlocked during the actual migration phase.

What do you think of this best-effort approach?

Best regards,
Wandun

The full reproducer is as below:

/* gcc repro.c -o repro -lpthread */

#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>

#define PAGE_SIZE       4096
#define NR_PAGES        32
#define FILE_SIZE       (2ULL * 1024 * 1024 * 1024)

static void *worker_fn(void *arg)

{
	int fd = (long)arg;
	size_t len = (size_t)FILE_SIZE;
	char *p = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
	if (p == MAP_FAILED)
		return NULL;

	for (size_t off = 0; off + NR_PAGES * PAGE_SIZE <= len;
	     off += NR_PAGES * PAGE_SIZE) {
		for (int i = 0; i < NR_PAGES; i++)
			p[off + i * PAGE_SIZE] = 1;
		usleep(200);
	}

	munmap(p, len);
	return NULL;
}

static void *compact_fn(void *arg)

{
	(void)arg;
	int fd = open("/proc/sys/vm/compact_memory", O_WRONLY);
	if (fd < 0)
		return NULL;

	while (1) {
		if (write(fd, "1", 1) < 0) {}
		usleep(5000);
	}
}

int main(void)

{
	mlockall(MCL_CURRENT | MCL_FUTURE);

	int fd = open("./repro_largefile.dat", O_RDWR | O_CREAT, 0600);
	if (fd < 0)
		return 1;
	unlink("./repro_largefile.dat");
	if (ftruncate(fd, (off_t)FILE_SIZE) < 0)
		return 1;

	printf("repro_largefile: 1 worker, %d pages/batch, Ctrl-C to stop\n",
	       NR_PAGES);

	pthread_t compact, worker;
	pthread_create(&compact, NULL, compact_fn, NULL);
	pthread_create(&worker, NULL, worker_fn, (void *)(long)fd);

	pthread_join(worker, NULL);
	return 0;
}
Quoted 19 lines
Quoted 18 lines
---
 mm/compaction.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/compaction.c +2 -1
index b776f35ad020..7e07b792bcb5 100644
@@ -1116,7 +1116,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		is_unevictable = folio_test_unevictable(folio);
 
 		/* Compaction might skip unevictable pages but CMA takes them */
-		if (!(mode & ISOLATE_UNEVICTABLE) && is_unevictable)
+		if (!(mode & ISOLATE_UNEVICTABLE) &&
+		    (is_unevictable || folio_test_mlocked(folio)))
 			goto isolate_fail_put;
 
 		/*
#4 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 37 lines · Wandun
Quoted 22 lines · Vlastimil Babka (SUSE)
Quoted 12 lines · Wandun Chen

From: Wandun Chen <[email protected]>

compact_unevictable_allowed is default 0 under PREEMPT_RT, isolate_migratepages_block() skips folios with PG_unevictable set. However, mlock_folio() sets PG_mlocked immediately but defers PG_unevictable to mlock_folio_batch(), result in a folio with PG_mlocked=1 but PG_unevictable=0. Compaction will isolate such a folio.

Fix by checking folio_test_mlocked() together with the existing folio_test_unevictable() check.

Quoted 8 lines

A similar issue has been reported by Alexander Krabler on a 6.12-rt aarch64 system. Vlastimil suggested to check the mlocked flag [1].

Quoted 4 lines

Well in that thread, Hugh doubted my suggestion and then it seems we didn't concluded anything. Did you actually in practice observe the issue that Alexander had, and that this patch fixed it, or is that theoretical?

Yes, I wrote a test case that can reproduce it in a few second.

The test case contains 3 steps:
1. mlockall
2. mmap file(2GB) + trigger file write page fault;
3. during step 1, trigger compact via /proc/sys/vm/compact_memory

Quoted 4 lines

My reproduction environment is qemu with 4GB ram, 8 core, aarch64, preempt_rt and includes the tracepoint in patch 02. After running the reproduction program for a few seconds, the following output appears.

Ah, nice.

Quoted 8 lines
repro-403     [004] ....1   101.270505: mm_compaction_isolate_folio: pfn=0x71e3a mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270507: mm_compaction_isolate_folio: pfn=0x71e3b mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270513: mm_compaction_isolate_folio: pfn=0x71e3c mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270515: mm_compaction_isolate_folio: pfn=0x71e3d mode=0x0 flags=uptodate|mlocked
repro-403     [004] ....1   101.270517: mm_compaction_isolate_folio: pfn=0x71e3e mode=0x0 flags=uptodate|mlocked
repro-403     [004] ....1   101.270520: mm_compaction_isolate_folio: pfn=0x71e3f mode=0x0 flags=uptodate|mlocked
Quoted 8 lines

Unfortunately, I recently found that there is still a bug in the fix patch. Setting mlocked in the mlock_folio function could happen even after the page is successfully isolated, so it still cannot prevent migration. Because of this, I need to think more about how to fix it.

Perhaps we should double-check whether the page is mlocked during the actual migration phase.

So IIUC the isolation+migration might be started between the folio is allocated, and mlocked? In that case the check during migration could still be racy, and if the page is isolated, it's already bad for the RT process.

So this would only be a short-term problem after the mlockall, but we don't have a way for the RT process to know the moment it's all settled, right? Probably the proper solution would be for mlock[all]() itself to wait for an isolated page, and only continue once it knows it can't be isolated anymore. This might howver would go against some of the folio batching optimizations?

Quoted 100 lines

What do you think of this best-effort approach?

Best regards,
Wandun

The full reproducer is as below:

/* gcc repro.c -o repro -lpthread */

#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>

#define PAGE_SIZE       4096
#define NR_PAGES        32
#define FILE_SIZE       (2ULL * 1024 * 1024 * 1024)

static void *worker_fn(void *arg)

{
	int fd = (long)arg;
	size_t len = (size_t)FILE_SIZE;
	char *p = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
	if (p == MAP_FAILED)
		return NULL;

	for (size_t off = 0; off + NR_PAGES * PAGE_SIZE <= len;
	     off += NR_PAGES * PAGE_SIZE) {
		for (int i = 0; i < NR_PAGES; i++)
			p[off + i * PAGE_SIZE] = 1;
		usleep(200);
	}

	munmap(p, len);
	return NULL;
}

static void *compact_fn(void *arg)

{
	(void)arg;
	int fd = open("/proc/sys/vm/compact_memory", O_WRONLY);
	if (fd < 0)
		return NULL;

	while (1) {
		if (write(fd, "1", 1) < 0) {}
		usleep(5000);
	}
}

int main(void)

{
	mlockall(MCL_CURRENT | MCL_FUTURE);

	int fd = open("./repro_largefile.dat", O_RDWR | O_CREAT, 0600);
	if (fd < 0)
		return 1;
	unlink("./repro_largefile.dat");
	if (ftruncate(fd, (off_t)FILE_SIZE) < 0)
		return 1;

	printf("repro_largefile: 1 worker, %d pages/batch, Ctrl-C to stop\n",
	       NR_PAGES);

	pthread_t compact, worker;
	pthread_create(&compact, NULL, compact_fn, NULL);
	pthread_create(&worker, NULL, worker_fn, (void *)(long)fd);

	pthread_join(worker, NULL);
	return 0;
}
Quoted 19 lines
Quoted 18 lines
---
 mm/compaction.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/compaction.c +2 -1
index b776f35ad020..7e07b792bcb5 100644
@@ -1116,7 +1116,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		is_unevictable = folio_test_unevictable(folio);
 
 		/* Compaction might skip unevictable pages but CMA takes them */
-		if (!(mode & ISOLATE_UNEVICTABLE) && is_unevictable)
+		if (!(mode & ISOLATE_UNEVICTABLE) &&
+		    (is_unevictable || folio_test_mlocked(folio)))
 			goto isolate_fail_put;
 
 		/*
author
#5 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 59 lines · Vlastimil Babka (SUSE)
Quoted 37 lines · Wandun
Quoted 22 lines · Vlastimil Babka (SUSE)
Quoted 12 lines · Wandun Chen

From: Wandun Chen <[email protected]>

compact_unevictable_allowed is default 0 under PREEMPT_RT, isolate_migratepages_block() skips folios with PG_unevictable set. However, mlock_folio() sets PG_mlocked immediately but defers PG_unevictable to mlock_folio_batch(), result in a folio with PG_mlocked=1 but PG_unevictable=0. Compaction will isolate such a folio.

Fix by checking folio_test_mlocked() together with the existing folio_test_unevictable() check.

Quoted 8 lines

A similar issue has been reported by Alexander Krabler on a 6.12-rt aarch64 system. Vlastimil suggested to check the mlocked flag [1].

Quoted 4 lines

Well in that thread, Hugh doubted my suggestion and then it seems we didn't concluded anything. Did you actually in practice observe the issue that Alexander had, and that this patch fixed it, or is that theoretical?

Yes, I wrote a test case that can reproduce it in a few second.

The test case contains 3 steps:
1. mlockall
2. mmap file(2GB) + trigger file write page fault;
3. during step 1, trigger compact via /proc/sys/vm/compact_memory

Quoted 4 lines

My reproduction environment is qemu with 4GB ram, 8 core, aarch64, preempt_rt and includes the tracepoint in patch 02. After running the reproduction program for a few seconds, the following output appears.

Ah, nice.

Quoted 14 lines
repro-403     [004] ....1   101.270505: mm_compaction_isolate_folio: pfn=0x71e3a mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270507: mm_compaction_isolate_folio: pfn=0x71e3b mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270513: mm_compaction_isolate_folio: pfn=0x71e3c mode=0x0 flags=referenced|uptodate|mlocked
repro-403     [004] ....1   101.270515: mm_compaction_isolate_folio: pfn=0x71e3d mode=0x0 flags=uptodate|mlocked
repro-403     [004] ....1   101.270517: mm_compaction_isolate_folio: pfn=0x71e3e mode=0x0 flags=uptodate|mlocked
repro-403     [004] ....1   101.270520: mm_compaction_isolate_folio: pfn=0x71e3f mode=0x0 flags=uptodate|mlocked

Unfortunately, I recently found that there is still a bug in the fix patch. Setting mlocked in the mlock_folio function could happen even after the page is successfully isolated, so it still cannot prevent migration. Because of this, I need to think more about how to fix it.

Quoted 5 lines
Quoted 2 lines

Perhaps we should double-check whether the page is mlocked during the actual migration phase.

So IIUC the isolation+migration might be started between the folio is allocated, and mlocked? In that case the check during migration could still

Yes, in that case it still be racy, it is not a good idea to check page flags.

Quoted 1 line

be racy, and if the page is isolated, it's already bad for the RT process.

IIUC, more accurately, the migration entry in the page talbe is real a bad for RT process, because isolate page doesn't modify the page table, so memory access continues as usual, therefore a new idea occur.

S1. In the mlock[all] syscall, if mlock_vma_pages_range hit a migration entry,
then, it should wait for the migration to complete.

S2. During the unmap phase of memory migration, prevent a page from being unmapped
if the page's associated vma is markd with VM_LOCKED, similar to how reclaim is disabled for pages in a VM_LOCKED vma(try_to_unmap_one).

For a page handled during the mlock[all] syscall:
- if migration has been already finished, there is noting to do;
- if migration is in progress and the migration etnry is already filled, we

    wait (S1)

- if the page is in-fight, going to be isolated/migrated, S2 prevents the unmap.

For a page handled during a page fault: VM_LOCKED is already set on the vma, so S2 guarantees it will not be unmapped, hence no migration entry.

Thanks a lot for the detailed feedback, Vlastimil.

Best regards,
Wandun

Quoted 3 lines

So this would only be a short-term problem after the mlockall, but we don't have a way for the RT process to know the moment it's all settled, right?

Yes, some pages may have been isolated and will do migration.

Quoted 105 lines

Probably the proper solution would be for mlock[all]() itself to wait for an isolated page, and only continue once it knows it can't be isolated anymore. This might howver would go against some of the folio batching optimizations?

Quoted 100 lines

What do you think of this best-effort approach?

Best regards,
Wandun

The full reproducer is as below:

/* gcc repro.c -o repro -lpthread */

#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>

#define PAGE_SIZE       4096
#define NR_PAGES        32
#define FILE_SIZE       (2ULL * 1024 * 1024 * 1024)

static void *worker_fn(void *arg)

{
	int fd = (long)arg;
	size_t len = (size_t)FILE_SIZE;
	char *p = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
	if (p == MAP_FAILED)
		return NULL;

	for (size_t off = 0; off + NR_PAGES * PAGE_SIZE <= len;
	     off += NR_PAGES * PAGE_SIZE) {
		for (int i = 0; i < NR_PAGES; i++)
			p[off + i * PAGE_SIZE] = 1;
		usleep(200);
	}

	munmap(p, len);
	return NULL;
}

static void *compact_fn(void *arg)

{
	(void)arg;
	int fd = open("/proc/sys/vm/compact_memory", O_WRONLY);
	if (fd < 0)
		return NULL;

	while (1) {
		if (write(fd, "1", 1) < 0) {}
		usleep(5000);
	}
}

int main(void)

{
	mlockall(MCL_CURRENT | MCL_FUTURE);

	int fd = open("./repro_largefile.dat", O_RDWR | O_CREAT, 0600);
	if (fd < 0)
		return 1;
	unlink("./repro_largefile.dat");
	if (ftruncate(fd, (off_t)FILE_SIZE) < 0)
		return 1;

	printf("repro_largefile: 1 worker, %d pages/batch, Ctrl-C to stop\n",
	       NR_PAGES);

	pthread_t compact, worker;
	pthread_create(&compact, NULL, compact_fn, NULL);
	pthread_create(&worker, NULL, worker_fn, (void *)(long)fd);

	pthread_join(worker, NULL);
	return 0;
}
Quoted 19 lines
Quoted 18 lines
---
 mm/compaction.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/compaction.c +2 -1
index b776f35ad020..7e07b792bcb5 100644
@@ -1116,7 +1116,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		is_unevictable = folio_test_unevictable(folio);
 
 		/* Compaction might skip unevictable pages but CMA takes them */
-		if (!(mode & ISOLATE_UNEVICTABLE) && is_unevictable)
+		if (!(mode & ISOLATE_UNEVICTABLE) &&
+		    (is_unevictable || folio_test_mlocked(folio)))
 			goto isolate_fail_put;
 
 		/*
#6 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 19 lines · Wandun
Quoted 9 lines · Vlastimil Babka (SUSE)
Quoted 8 lines · Wandun

Yes, I wrote a test case that can reproduce it in a few second.

The test case contains 3 steps:
1. mlockall
2. mmap file(2GB) + trigger file write page fault;
3. during step 1, trigger compact via /proc/sys/vm/compact_memory

Quoted 6 lines
Quoted 6 lines

My reproduction environment is qemu with 4GB ram, 8 core, aarch64, preempt_rt and includes the tracepoint in patch 02. After running the reproduction program for a few seconds, the following output appears.

repro-403 [004] ....1 101.270505: mm_compaction_isolate_folio: pfn=0x71e3a mode=0x0

flags=referenced|uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270507: mm_compaction_isolate_folio: pfn=0x71e3b mode=0x0

flags=referenced|uptodate|mlocked

Quoted 8 lines
Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270513: mm_compaction_isolate_folio: pfn=0x71e3c mode=0x0

flags=referenced|uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270515: mm_compaction_isolate_folio: pfn=0x71e3d mode=0x0

flags=uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270517: mm_compaction_isolate_folio: pfn=0x71e3e mode=0x0

flags=uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270520: mm_compaction_isolate_folio: pfn=0x71e3f mode=0x0

flags=uptodate|mlocked

I applied your PATCH 2/3 to our kernel and checked with your reproducer,
I get similar output, e.g.
t_compact-2148 [005] ....1 515.320221: mm_compaction_isolate_folio: pfn=0xe66c2 mode=0x0
flags=referenced|uptodate|active|swapbacked|mlocked

With your first patch applied, the amount of these messages decrease. I was not able to apply your third patch to our (older) kernel.

However, we were not able to reproduce the actual race (mlockall() process waiting on a migration PTE), not in the past, not now. Might be hard to trigger that race.

Quoted 12 lines

IIUC, more accurately, the migration entry in the page talbe is real a bad for RT process, because isolate page doesn't modify the page table, so memory access continues as usual, therefore a new idea occur.

S1. In the mlock[all] syscall, if mlock_vma_pages_range hit a migration entry,
then, it should wait for the migration to complete.

S2. During the unmap phase of memory migration, prevent a page from being unmapped
if the page's associated vma is markd with VM_LOCKED, similar to how reclaim is disabled for pages in a VM_LOCKED vma(try_to_unmap_one).

Quoted 8 lines

For a page handled during the mlock[all] syscall:
- if migration has been already finished, there is noting to do;
- if migration is in progress and the migration etnry is already filled, we

    wait (S1)

- if the page is in-fight, going to be isolated/migrated, S2 prevents the unmap.

For a page handled during a page fault: VM_LOCKED is already set on the vma, so S2 guarantees it will not be unmapped, hence no migration entry.

I do not understand all details of this, but it looks good, especially the S1 case makes a lot of sense for me.

Nitpick: I suggest to switch order of PATCH 1 and 2 for the next iteration, introducing the tracepoint first and then improve the situation.

Thanks a lot for looking into this issue!

Best regards,
Alexander

--

KUKA Deutschland GmbH Board of Directors: Michael Jürgens (Chairman), Johan Naten, Hui Zhang Registered Office: Augsburg HRB 14914

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of contents of this e-mail is strictly forbidden.

Please consider the environment before printing this e-mail.

author
#7 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 27 lines · Alexander Krabler
Quoted 26 lines · Wandun
Quoted 9 lines · Vlastimil Babka (SUSE)
Quoted 8 lines · Wandun

Yes, I wrote a test case that can reproduce it in a few second.

The test case contains 3 steps:
1. mlockall
2. mmap file(2GB) + trigger file write page fault;
3. during step 1, trigger compact via /proc/sys/vm/compact_memory

Quoted 6 lines
Quoted 6 lines

My reproduction environment is qemu with 4GB ram, 8 core, aarch64, preempt_rt and includes the tracepoint in patch 02. After running the reproduction program for a few seconds, the following output appears.

repro-403 [004] ....1 101.270505: mm_compaction_isolate_folio: pfn=0x71e3a mode=0x0

flags=referenced|uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270507: mm_compaction_isolate_folio: pfn=0x71e3b mode=0x0

flags=referenced|uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270513: mm_compaction_isolate_folio: pfn=0x71e3c mode=0x0

flags=referenced|uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270515: mm_compaction_isolate_folio: pfn=0x71e3d mode=0x0

flags=uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270517: mm_compaction_isolate_folio: pfn=0x71e3e mode=0x0

flags=uptodate|mlocked

Quoted 1 line
Quoted 1 line

repro-403 [004] ....1 101.270520: mm_compaction_isolate_folio: pfn=0x71e3f mode=0x0

Quoted 8 lines
Quoted 1 line

flags=uptodate|mlocked

I applied your PATCH 2/3 to our kernel and checked with your reproducer,
I get similar output, e.g.
t_compact-2148 [005] ....1 515.320221: mm_compaction_isolate_folio: pfn=0xe66c2 mode=0x0
flags=referenced|uptodate|active|swapbacked|mlocked

With your first patch applied, the amount of these messages decrease.

Parts of mlocked but not unevictable pages has been filter out, so messages decrease, but racy is still there.

Quoted 1 line

I was not able to apply your third patch to our (older) kernel.

Patch 3 is meaningless to you. The problem in your report is caused by kcompactd, not cma alloc, so it is of no use to you.

Quoted 4 lines

However, we were not able to reproduce the actual race (mlockall() process waiting on a migration PTE), not in the past, not now. Might be hard to trigger that race.

Not hard to trigger that case, I added a debug message, such as below, lots of messages occur in a few second.

mm/memory.c +0 -0
diff --cc mm/memory.c
index ff338c2abe92,ff338c2abe92..6552b3b14f78
@@@ -4768,6 -4768,6 +4768,8 @@@ vm_fault_t do_swap_page(struct vm_faul
                if (softleaf_is_migration(entry)) {
                        migration_entry_wait(vma->vm_mm, vmf->pmd,
                                             vmf->address);
+                       if (!strcmp(current->comm, "repro"))
+                               pr_err("============== hit ================\n");
                } else if (softleaf_is_device_exclusive(entry)) {
                        vmf->page = softleaf_to_page(entry);
                        ret = remove_device_exclusive_entry(vmf);

Best regard,
Wandun

Quoted 40 lines
Quoted 13 lines

IIUC, more accurately, the migration entry in the page talbe is real a bad for RT process, because isolate page doesn't modify the page table, so memory access continues as usual, therefore a new idea occur.

S1. In the mlock[all] syscall, if mlock_vma_pages_range hit a migration entry,
then, it should wait for the migration to complete.

S2. During the unmap phase of memory migration, prevent a page from being unmapped
if the page's associated vma is markd with VM_LOCKED, similar to how reclaim is disabled for pages in a VM_LOCKED vma(try_to_unmap_one).

Quoted 8 lines

For a page handled during the mlock[all] syscall:
- if migration has been already finished, there is noting to do;
- if migration is in progress and the migration etnry is already filled, we

    wait (S1)

- if the page is in-fight, going to be isolated/migrated, S2 prevents the unmap.

For a page handled during a page fault: VM_LOCKED is already set on the vma, so S2 guarantees it will not be unmapped, hence no migration entry.

I do not understand all details of this, but it looks good, especially the S1 case makes a lot of sense for me.

Nitpick: I suggest to switch order of PATCH 1 and 2 for the next iteration, introducing the tracepoint first and then improve the situation.

Thanks a lot for looking into this issue!

Best regards,
Alexander

--

KUKA Deutschland GmbH Board of Directors: Michael Jürgens (Chairman), Johan Naten, Hui Zhang Registered Office: Augsburg HRB 14914

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of contents of this e-mail is strictly forbidden.

Please consider the environment before printing this e-mail.

#8 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 13 lines · Wandun
Quoted 3 lines · Alexander Krabler

However, we were not able to reproduce the actual race (mlockall() process waiting on a migration PTE), not in the past, not now. Might be hard to trigger that race.

Not hard to trigger that case, I added a debug message, such as below, lots of messages occur in a few second.

mm/memory.c +0 -0
diff --cc mm/memory.c
index ff338c2abe92,ff338c2abe92..6552b3b14f78

mm/memory.c

Quoted 8 lines
mm/memory.c +2 -0
@@@ -4768,6 -4768,6 +4768,8 @@@ vm_fault_t do_swap_page(struct vm_faul
                if (softleaf_is_migration(entry)) {
                        migration_entry_wait(vma->vm_mm, vmf->pmd,
                                             vmf->address);
+                       if (!strcmp(current->comm, "repro"))
+                               pr_err("============== hit ================\n");
                } else if (softleaf_is_device_exclusive(entry)) {
                        vmf->page = softleaf_to_page(entry);
                        ret = remove_device_exclusive_entry(vmf);

I have a kprobe on migration_entry_wait set and logged into a ftrace buffer (including kernel stacktrace).
Yes, this function is hit, but only inside the mmap-syscall, which is okay, memory allocation is not realtime-safe.

           repro-2090    [002] d....   811.129549: frt_migration_entry_wait: (migration_entry_wait+0x0/0x100)
           repro-2090    [002] d....   811.129553: <stack trace>

=> migration_entry_wait
=> __handle_mm_fault
=> handle_mm_fault
=> __get_user_pages
=> populate_vma_page_range
=> __mm_populate
=> vm_mmap_pgoff
=> ksys_mmap_pgoff
=> __arm64_sys_mmap
=> el0_svc_common.constprop.0
=> do_el0_svc
=> el0_svc
=> el0t_64_sync_handler
=> el0t_64_sync

The original race was an instruction abort interrupt out of nothing due to the migration PTE set by kcompactd.
And these kind of races I see quite often on non mlockall()-processes, but can't reproduce on memory locked processes.

Example:

          podman-832     [000] d....   812.447820: frt_migration_entry_wait: (migration_entry_wait+0x0/0x100)
          podman-832     [000] d....   812.447823: <stack trace>

=> migration_entry_wait
=> __handle_mm_fault
=> handle_mm_fault
=> do_page_fault
=> do_translation_fault
=> do_mem_abort
=> el0_da
=> el0t_64_sync_handler
=> el0t_64_sync

Thanks,
Alexander

--

KUKA Deutschland GmbH Board of Directors: Michael Jürgens (Chairman), Johan Naten, Hui Zhang Registered Office: Augsburg HRB 14914

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of contents of this e-mail is strictly forbidden.

Please consider the environment before printing this e-mail.

author
#9 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 53 lines · Alexander Krabler
Quoted 13 lines · Wandun
Quoted 3 lines · Alexander Krabler

However, we were not able to reproduce the actual race (mlockall() process waiting on a migration PTE), not in the past, not now. Might be hard to trigger that race.

Not hard to trigger that case, I added a debug message, such as below, lots of messages occur in a few second.

mm/memory.c +0 -0
diff --cc mm/memory.c
index ff338c2abe92,ff338c2abe92..6552b3b14f78

mm/memory.c

Quoted 8 lines
mm/memory.c +2 -0
@@@ -4768,6 -4768,6 +4768,8 @@@ vm_fault_t do_swap_page(struct vm_faul
                if (softleaf_is_migration(entry)) {
                        migration_entry_wait(vma->vm_mm, vmf->pmd,
                                             vmf->address);
+                       if (!strcmp(current->comm, "repro"))
+                               pr_err("============== hit ================\n");
                } else if (softleaf_is_device_exclusive(entry)) {
                        vmf->page = softleaf_to_page(entry);
                        ret = remove_device_exclusive_entry(vmf);

I have a kprobe on migration_entry_wait set and logged into a ftrace buffer (including kernel stacktrace).
Yes, this function is hit, but only inside the mmap-syscall, which is okay, memory allocation is not realtime-safe.

           repro-2090    [002] d....   811.129549: frt_migration_entry_wait: (migration_entry_wait+0x0/0x100)
           repro-2090    [002] d....   811.129553: <stack trace>

=> migration_entry_wait
=> __handle_mm_fault
=> handle_mm_fault
=> __get_user_pages
=> populate_vma_page_range
=> __mm_populate
=> vm_mmap_pgoff
=> ksys_mmap_pgoff
=> __arm64_sys_mmap
=> el0_svc_common.constprop.0
=> do_el0_svc
=> el0_svc
=> el0t_64_sync_handler
=> el0t_64_sync

The original race was an instruction abort interrupt out of nothing due to the migration PTE set by kcompactd.
And these kind of races I see quite often on non mlockall()-processes, but can't reproduce on memory locked processes.

Example:

          podman-832     [000] d....   812.447820: frt_migration_entry_wait: (migration_entry_wait+0x0/0x100)
          podman-832     [000] d....   812.447823: <stack trace>
Quoted 9 lines

=> migration_entry_wait
=> __handle_mm_fault
=> handle_mm_fault
=> do_page_fault
=> do_translation_fault
=> do_mem_abort
=> el0_da
=> el0t_64_sync_handler
=> el0t_64_sync

Hi, Alexander

From the perspective of the root cause, there is no fundamental difference between these two call stacks. I modified the reproduction program, and it can still reproduce the situation of the second call stack (although it doesn't occur as frequently). The complete reproduction program is as follows:

#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/sysinfo.h>
#include <unistd.h>

#define PAGE_SIZE 4096
#define NR_PAGES 10000

static void *worker_fn(void *arg)

{
	int fd = (long)arg;
	size_t len = NR_PAGES * PAGE_SIZE;

	while (1) {
		if (ftruncate(fd, 0) < 0) {}
		if (ftruncate(fd, len) < 0) {}

		char *p = mmap(NULL, len, PROT_READ | PROT_WRITE,
			       MAP_SHARED, fd, 0);
		if (p == MAP_FAILED)
			continue;

		mlockall(MCL_ONFAULT |  MCL_FUTURE);

		for (int i = 0; i < NR_PAGES; i++) {
			for (int j = 0; j < PAGE_SIZE; j++) {
				p[i * PAGE_SIZE + j] = 1;
			}
		}

		usleep(200);
		munmap(p, len);
	}
	return NULL;
}

static void *compact_fn(void *arg)

{
	(void)arg;
	int fd = open("/proc/sys/vm/compact_memory", O_WRONLY);
	if (fd < 0)
		return NULL;

	while (1) {
		if (write(fd, "1", 1) < 0) {}
		usleep(5000);
	}
}

int main(void)

{
	int nproc = sysconf(_SC_NPROCESSORS_ONLN);
	if (nproc < 1)
		nproc = 1;

	int *fds = calloc((size_t)nproc, sizeof(int));
	if (!fds)
		return 1;

	size_t len = NR_PAGES * PAGE_SIZE;
	for (int i = 0; i < nproc; i++) {
		char path[64];
		snprintf(path, sizeof(path), "./repro_%d.dat", i);
		unlink(path);
		fds[i] = open(path, O_RDWR | O_CREAT, 0600);
		if (fds[i] < 0)
			return 1;
		if (ftruncate(fds[i], len) < 0)
			return 1;
	}

	printf("repro: %d workers, %d pages, Ctrl-C to stop\n",
	       nproc, NR_PAGES);

	pthread_t compact;
	pthread_create(&compact, NULL, compact_fn, NULL);

	pthread_t *threads = calloc((size_t)nproc, sizeof(pthread_t));
	for (int i = 0; i < nproc; i++)
		pthread_create(&threads[i], NULL, worker_fn, (void *)(long)fds[i]);

	pthread_join(compact, NULL);
	return 0;
}
Quoted 11 lines

Thanks,
Alexander

--

KUKA Deutschland GmbH Board of Directors: Michael Jürgens (Chairman), Johan Naten, Hui Zhang Registered Office: Augsburg HRB 14914

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of contents of this e-mail is strictly forbidden.

Please consider the environment before printing this e-mail.

author
#10 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 156 lines · Wandun
Quoted 53 lines · Alexander Krabler
Quoted 13 lines · Wandun
Quoted 3 lines · Alexander Krabler

However, we were not able to reproduce the actual race (mlockall() process waiting on a migration PTE), not in the past, not now. Might be hard to trigger that race.

Not hard to trigger that case, I added a debug message, such as below, lots of messages occur in a few second.

mm/memory.c +0 -0
diff --cc mm/memory.c
index ff338c2abe92,ff338c2abe92..6552b3b14f78

mm/memory.c

Quoted 8 lines
mm/memory.c +2 -0
@@@ -4768,6 -4768,6 +4768,8 @@@ vm_fault_t do_swap_page(struct vm_faul
                if (softleaf_is_migration(entry)) {
                        migration_entry_wait(vma->vm_mm, vmf->pmd,
                                             vmf->address);
+                       if (!strcmp(current->comm, "repro"))
+                               pr_err("============== hit ================\n");
                } else if (softleaf_is_device_exclusive(entry)) {
                        vmf->page = softleaf_to_page(entry);
                        ret = remove_device_exclusive_entry(vmf);

I have a kprobe on migration_entry_wait set and logged into a ftrace buffer (including kernel stacktrace).
Yes, this function is hit, but only inside the mmap-syscall, which is okay, memory allocation is not realtime-safe.

           repro-2090    [002] d....   811.129549: frt_migration_entry_wait: (migration_entry_wait+0x0/0x100)
           repro-2090    [002] d....   811.129553: <stack trace>

=> migration_entry_wait
=> __handle_mm_fault
=> handle_mm_fault
=> __get_user_pages
=> populate_vma_page_range
=> __mm_populate
=> vm_mmap_pgoff
=> ksys_mmap_pgoff
=> __arm64_sys_mmap
=> el0_svc_common.constprop.0
=> do_el0_svc
=> el0_svc
=> el0t_64_sync_handler
=> el0t_64_sync

The original race was an instruction abort interrupt out of nothing due to the migration PTE set by kcompactd.
And these kind of races I see quite often on non mlockall()-processes, but can't reproduce on memory locked processes.

Example:

          podman-832     [000] d....   812.447820: frt_migration_entry_wait: (migration_entry_wait+0x0/0x100)
          podman-832     [000] d....   812.447823: <stack trace>
Quoted 9 lines

=> migration_entry_wait
=> __handle_mm_fault
=> handle_mm_fault
=> do_page_fault
=> do_translation_fault
=> do_mem_abort
=> el0_da
=> el0t_64_sync_handler
=> el0t_64_sync

Hi, Alexander

From the perspective of the root cause, there is no fundamental difference between these two call stacks. I modified the reproduction program, and it can still reproduce the situation of the second call stack (although it doesn't occur as frequently). The complete reproduction program is as follows:

#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/sysinfo.h>
#include <unistd.h>

#define PAGE_SIZE 4096
#define NR_PAGES 10000

static void *worker_fn(void *arg)

{
	int fd = (long)arg;
	size_t len = NR_PAGES * PAGE_SIZE;

	while (1) {
		if (ftruncate(fd, 0) < 0) {}
		if (ftruncate(fd, len) < 0) {}

		char *p = mmap(NULL, len, PROT_READ | PROT_WRITE,
			       MAP_SHARED, fd, 0);
		if (p == MAP_FAILED)
			continue;

		mlockall(MCL_ONFAULT |  MCL_FUTURE);

		for (int i = 0; i < NR_PAGES; i++) {
			for (int j = 0; j < PAGE_SIZE; j++) {
				p[i * PAGE_SIZE + j] = 1;
			}
		}

		usleep(200);
		munmap(p, len);
	}
	return NULL;
}

static void *compact_fn(void *arg)

{
	(void)arg;
	int fd = open("/proc/sys/vm/compact_memory", O_WRONLY);
	if (fd < 0)
		return NULL;

	while (1) {
		if (write(fd, "1", 1) < 0) {}
		usleep(5000);
	}
}

int main(void)

{
	int nproc = sysconf(_SC_NPROCESSORS_ONLN);
	if (nproc < 1)
		nproc = 1;

	int *fds = calloc((size_t)nproc, sizeof(int));
	if (!fds)
		return 1;

	size_t len = NR_PAGES * PAGE_SIZE;
	for (int i = 0; i < nproc; i++) {
		char path[64];
		snprintf(path, sizeof(path), "./repro_%d.dat", i);
		unlink(path);
		fds[i] = open(path, O_RDWR | O_CREAT, 0600);
		if (fds[i] < 0)
			return 1;
		if (ftruncate(fds[i], len) < 0)
			return 1;
	}

	printf("repro: %d workers, %d pages, Ctrl-C to stop\n",
	       nproc, NR_PAGES);

	pthread_t compact;
	pthread_create(&compact, NULL, compact_fn, NULL);
Quoted 7 lines
	pthread_t *threads = calloc((size_t)nproc, sizeof(pthread_t));
	for (int i = 0; i < nproc; i++)
		pthread_create(&threads[i], NULL, worker_fn, (void *)(long)fds[i]);

	pthread_join(compact, NULL);
	return 0;
}

I found there are false positives in this reproducer. I modified the program a little, the diff is as follows, the problem can still be reproduced

repro.c +1 -2
index b1b34d5..59cb417 100644
@@ -15,6 +15,7 @@ static void *worker_fn(void *arg)
        int fd = (long)arg;
        size_t len = NR_PAGES * PAGE_SIZE;
 
+       mlockall(MCL_ONFAULT |  MCL_FUTURE | MCL_CURRENT);
        while (1) {
                if (ftruncate(fd, 0) < 0) {}
                if (ftruncate(fd, len) < 0) {}
@@ -24,8 +25,6 @@ static void *worker_fn(void *arg)
                if (p == MAP_FAILED)
                        continue;
 
-               mlockall(MCL_ONFAULT |  MCL_FUTURE);
-
                for (int i = 0; i < NR_PAGES; i++) {
                        for (int j = 0; j < PAGE_SIZE; j++) {
                                p[i * PAGE_SIZE + j] = 1;
Quoted 16 lines
Quoted 15 lines

Thanks,
Alexander

--

KUKA Deutschland GmbH Board of Directors: Michael Jürgens (Chairman), Johan Naten, Hui Zhang Registered Office: Augsburg HRB 14914

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of contents of this e-mail is strictly forbidden.

Please consider the environment before printing this e-mail.

#11 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 18 lines · Wandun

I found there are false positives in this reproducer. I modified the program a little, the diff is as follows, the problem can still be reproduced

repro.c +1 -0
index b1b34d5..59cb417 100644
@@ -15,6 +15,7 @@ static void *worker_fn(void *arg)
        int fd = (long)arg;
        size_t len = NR_PAGES * PAGE_SIZE;
 
+       mlockall(MCL_ONFAULT |  MCL_FUTURE | MCL_CURRENT);
        while (1) {
                if (ftruncate(fd, 0) < 0) {}
                if (ftruncate(fd, len) < 0) {}

repro.c static void *worker_fn(void *arg) · line 29

Quoted 7 lines
repro.c +0 -2
@@ -24,8 +25,6 @@ static void *worker_fn(void *arg)
                if (p == MAP_FAILED)
                        continue;
 
-               mlockall(MCL_ONFAULT |  MCL_FUTURE);
-
                for (int i = 0; i < NR_PAGES; i++) {
                        for (int j = 0; j < PAGE_SIZE; j++) {

No worries, I can reproduce with that program, but I still think this is different.
The reproducer passes the MCL_ONFAULT flag to mlockall(), delaying the actual memory allocation.

However, this is not what I would do in realtime programs and if I do, these faults are basically memory allocations, where I can not expect realtime performance guarantees.

If I remove MCL_ONFAULT, I did not manage to hit the migration_entry_wait with the reproducer.

My guess is, that it only triggers when using CMA, but I'm not deep enough into that topic to proof that.

--

KUKA Deutschland GmbH Board of Directors: Michael Jürgens (Chairman), Johan Naten, Hui Zhang Registered Office: Augsburg HRB 14914

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of contents of this e-mail is strictly forbidden.

Please consider the environment before printing this e-mail.

#12 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

In reply to Wandun Chen

On 2026-06-04 10:38:10 [+0800], Wandun Chen wrote:
…

Quoted 5 lines

Is it possible to get a Fixes tag on the final fix so that it can be backported stable?

Sebastian

author
#13 lore raw

Re: [RFC PATCH 1/3] mm/compaction: skip isolate mlocked folios when compact_unevictable_allowed=0

Quoted 10 lines · Sebastian Andrzej Siewior

On 2026-06-04 10:38:10 [+0800], Wandun Chen wrote:
…

Quoted 5 lines

Is it possible to get a Fixes tag on the final fix so that it can be backported stable?

Got it.

Best regards,
Wandun

Quoted 2 lines

Sebastian

#14 lore raw

RFC PATCH 2/3 mm/compaction: add per-folio isolation tracepoint

In reply to Wandun Chen

From: Wandun Chen <[email protected]>

Add a tracepoint that fires once per folio successfully isolated by isolate_migratepages_block(), recording the pfn, isolation mode and the folio flags. Knowing these makes it easier to debug unexpected isolation, such as mlocked or unevictable folios showing up on PREEMPT_RT kernels [1].

---
 include/trace/events/compaction.h | 26 ++++++++++++++++++++++++++
 mm/compaction.c                   |  2 ++
 2 files changed, 28 insertions(+)
include/trace/events/compaction.h +26 -0
index d05759d18538..8b8b3ec0f324 100644
@@ -76,6 +76,32 @@ DEFINE_EVENT(mm_compaction_isolate_template, mm_compaction_fast_isolate_freepage
 );
 
 #ifdef CONFIG_COMPACTION
+TRACE_EVENT(mm_compaction_isolate_folio,
+
+	TP_PROTO(unsigned long pfn,
+		isolate_mode_t mode,
+		unsigned long flags),
+
+	TP_ARGS(pfn, mode, flags),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, pfn)
+		__field(isolate_mode_t, mode)
+		__field(unsigned long, flags)
+	),
+
+	TP_fast_assign(
+		__entry->pfn = pfn;
+		__entry->mode = mode;
+		__entry->flags = flags;
+	),
+
+	TP_printk("pfn=0x%lx mode=0x%x flags=%s",
+		__entry->pfn,
+		__entry->mode,
+		show_page_flags(__entry->flags & PAGEFLAGS_MASK))
+);
+
 TRACE_EVENT(mm_compaction_migratepages,
 
 	TP_PROTO(unsigned int nr_migratepages,
mm/compaction.c +2 -0
index 7e07b792bcb5..007d5e00a8ae 100644
@@ -1223,6 +1223,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 isolate_success:
 		list_add(&folio->lru, &cc->migratepages);
 isolate_success_no_list:
+		trace_mm_compaction_isolate_folio(folio_pfn(folio), mode,
+						  folio->flags.f);
 		cc->nr_migratepages += folio_nr_pages(folio);
 		nr_isolated += folio_nr_pages(folio);
 		nr_scanned += folio_nr_pages(folio) - 1;
Signature · 1 line
2.43.0
#15 lore raw

RFC PATCH 3/3 mm/compaction: respect compact_unevictable_allowed in alloc_contig path

In reply to Wandun Chen

From: Wandun Chen <[email protected]>

vm.compact_unevictable_allowed=0 is used to prevent compacting unevictable pages. However, isolate_migratepages_range() passes ISOLATE_UNEVICTABLE regardless of this sysctl, so the setting has no effect in the alloc_contig path.

Fix it by:
- Keep ISOLATE_UNEVICTABLE for CMA allocation, discussed in [1].
- Honour sysctl_compact_unevictable_allowed for non-CMA allocation.

---
 include/linux/compaction.h | 6 ++++++
 mm/compaction.c            | 9 +++++++--
 mm/internal.h              | 1 +
 mm/page_alloc.c            | 2 ++
 4 files changed, 16 insertions(+), 2 deletions(-)
include/linux/compaction.h +6 -0
index f29ef0653546..04e60f65b976 100644
@@ -106,6 +106,7 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
 extern void __meminit kcompactd_run(int nid);
 extern void __meminit kcompactd_stop(int nid);
 extern void wakeup_kcompactd(pg_data_t *pgdat, int order, int highest_zoneidx);
+extern bool compaction_allow_unevictable(void);
 
 #else
 static inline void reset_isolation_suitable(pg_data_t *pgdat)
@@ -131,6 +132,11 @@ static inline void wakeup_kcompactd(pg_data_t *pgdat,
 {
 }
 
+static inline bool compaction_allow_unevictable(void)
+{
+	return true;
+}
+
 #endif /* CONFIG_COMPACTION */
 
 struct node;
mm/compaction.c +7 -2
index 007d5e00a8ae..a10acb273454 100644
@@ -1341,6 +1341,7 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn,
 							unsigned long end_pfn)
 {
 	unsigned long pfn, block_start_pfn, block_end_pfn;
+	isolate_mode_t mode = cc->allow_unevictable ? ISOLATE_UNEVICTABLE : 0;
 	int ret = 0;
 
 	/* Scan block by block. First and last block may be incomplete */
@@ -1360,8 +1361,7 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn,
 					block_end_pfn, cc->zone))
 			continue;
 
-		ret = isolate_migratepages_block(cc, pfn, block_end_pfn,
-						 ISOLATE_UNEVICTABLE);
+		ret = isolate_migratepages_block(cc, pfn, block_end_pfn, mode);
 
 		if (ret)
 			break;
@@ -1902,6 +1902,11 @@ typedef enum {
  * compactable pages.
  */
 static int sysctl_compact_unevictable_allowed __read_mostly = CONFIG_COMPACT_UNEVICTABLE_DEFAULT;
+
+bool compaction_allow_unevictable(void)
+{
+	return sysctl_compact_unevictable_allowed;
+}
 /*
  * Tunable for proactive compaction. It determines how
  * aggressively the kernel should compact memory in the
mm/internal.h +1 -0
index 181e79f1d6a2..163f9d6b37f3 100644
@@ -1052,6 +1052,7 @@ struct compact_control {
 					 * ensure forward progress.
 					 */
 	bool alloc_contig;		/* alloc_contig_range allocation */
+	bool allow_unevictable;		/* Allow isolation of unevictable folios */
 };
 
 /*
mm/page_alloc.c +2 -0
index 81a9d4d1e6c0..1cf9d4a3b14c 100644
@@ -7118,6 +7118,8 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 		.ignore_skip_hint = true,
 		.no_set_skip_hint = true,
 		.alloc_contig = true,
+		.allow_unevictable = !!(alloc_flags & ACR_FLAGS_CMA) ||
+					     compaction_allow_unevictable(),
 	};
 	INIT_LIST_HEAD(&cc.migratepages);
 	enum pb_isolate_mode mode = (alloc_flags & ACR_FLAGS_CMA) ?
Signature · 1 line
2.43.0
#16 lore raw

Re: [RFC PATCH 3/3] mm/compaction: respect compact_unevictable_allowed in alloc_contig path

Quoted 7 lines · Wandun Chen

From: Wandun Chen <[email protected]>

vm.compact_unevictable_allowed=0 is used to prevent compacting unevictable pages. However, isolate_migratepages_range() passes ISOLATE_UNEVICTABLE regardless of this sysctl, so the setting has no effect in the alloc_contig path.

Quoted 7 lines

Fix it by:
- Keep ISOLATE_UNEVICTABLE for CMA allocation, discussed in [1].
- Honour sysctl_compact_unevictable_allowed for non-CMA allocation.

There was also the "Ideally by not having mlock'd pages in CMA areas at all." part. Is it the case? It was more elaborated here:
https://lore.kernel.org/all/CAPTztWZpnX1j8-7yeppVUsxE=O9hbVeqricDjZt8_pnN7a-kBQ@mail.gmail.com/

Quoted 90 lines
---
 include/linux/compaction.h | 6 ++++++
 mm/compaction.c            | 9 +++++++--
 mm/internal.h              | 1 +
 mm/page_alloc.c            | 2 ++
 4 files changed, 16 insertions(+), 2 deletions(-)
include/linux/compaction.h +6 -0
index f29ef0653546..04e60f65b976 100644
@@ -106,6 +106,7 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
 extern void __meminit kcompactd_run(int nid);
 extern void __meminit kcompactd_stop(int nid);
 extern void wakeup_kcompactd(pg_data_t *pgdat, int order, int highest_zoneidx);
+extern bool compaction_allow_unevictable(void);
 
 #else
 static inline void reset_isolation_suitable(pg_data_t *pgdat)
@@ -131,6 +132,11 @@ static inline void wakeup_kcompactd(pg_data_t *pgdat,
 {
 }
 
+static inline bool compaction_allow_unevictable(void)
+{
+	return true;
+}
+
 #endif /* CONFIG_COMPACTION */
 
 struct node;
mm/compaction.c +7 -2
index 007d5e00a8ae..a10acb273454 100644
@@ -1341,6 +1341,7 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn,
 							unsigned long end_pfn)
 {
 	unsigned long pfn, block_start_pfn, block_end_pfn;
+	isolate_mode_t mode = cc->allow_unevictable ? ISOLATE_UNEVICTABLE : 0;
 	int ret = 0;
 
 	/* Scan block by block. First and last block may be incomplete */
@@ -1360,8 +1361,7 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn,
 					block_end_pfn, cc->zone))
 			continue;
 
-		ret = isolate_migratepages_block(cc, pfn, block_end_pfn,
-						 ISOLATE_UNEVICTABLE);
+		ret = isolate_migratepages_block(cc, pfn, block_end_pfn, mode);
 
 		if (ret)
 			break;
@@ -1902,6 +1902,11 @@ typedef enum {
  * compactable pages.
  */
 static int sysctl_compact_unevictable_allowed __read_mostly = CONFIG_COMPACT_UNEVICTABLE_DEFAULT;
+
+bool compaction_allow_unevictable(void)
+{
+	return sysctl_compact_unevictable_allowed;
+}
 /*
  * Tunable for proactive compaction. It determines how
  * aggressively the kernel should compact memory in the
mm/internal.h +1 -0
index 181e79f1d6a2..163f9d6b37f3 100644
@@ -1052,6 +1052,7 @@ struct compact_control {
 					 * ensure forward progress.
 					 */
 	bool alloc_contig;		/* alloc_contig_range allocation */
+	bool allow_unevictable;		/* Allow isolation of unevictable folios */
 };
 
 /*
mm/page_alloc.c +2 -0
index 81a9d4d1e6c0..1cf9d4a3b14c 100644
@@ -7118,6 +7118,8 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 		.ignore_skip_hint = true,
 		.no_set_skip_hint = true,
 		.alloc_contig = true,
+		.allow_unevictable = !!(alloc_flags & ACR_FLAGS_CMA) ||
+					     compaction_allow_unevictable(),
 	};
 	INIT_LIST_HEAD(&cc.migratepages);
 	enum pb_isolate_mode mode = (alloc_flags & ACR_FLAGS_CMA) ?
author
#17 lore raw

Re: [RFC PATCH 3/3] mm/compaction: respect compact_unevictable_allowed in alloc_contig path

Quoted 12 lines · Vlastimil Babka (SUSE)
Quoted 11 lines · Wandun Chen

From: Wandun Chen <[email protected]>

vm.compact_unevictable_allowed=0 is used to prevent compacting unevictable pages. However, isolate_migratepages_range() passes ISOLATE_UNEVICTABLE regardless of this sysctl, so the setting has no effect in the alloc_contig path.

Fix it by:
- Keep ISOLATE_UNEVICTABLE for CMA allocation, discussed in [1].
- Honour sysctl_compact_unevictable_allowed for non-CMA allocation.

Quoted 6 lines
Quoted 3 lines

There was also the "Ideally by not having mlock'd pages in CMA areas at all." part. Is it the case? It was more elaborated here:

Yes, It is the case.

Quoted 1 line

https://lore.kernel.org/all/CAPTztWZpnX1j8-7yeppVUsxE=O9hbVeqricDjZt8_pnN7a-kBQ@mail.gmail.com/

I missed this important information. Thanks for pointing it out, Vlastimil.

Best regards,
Wandun

Quoted 92 lines
Quoted 91 lines
---
 include/linux/compaction.h | 6 ++++++
 mm/compaction.c            | 9 +++++++--
 mm/internal.h              | 1 +
 mm/page_alloc.c            | 2 ++
 4 files changed, 16 insertions(+), 2 deletions(-)
include/linux/compaction.h +6 -0
index f29ef0653546..04e60f65b976 100644
@@ -106,6 +106,7 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
 extern void __meminit kcompactd_run(int nid);
 extern void __meminit kcompactd_stop(int nid);
 extern void wakeup_kcompactd(pg_data_t *pgdat, int order, int highest_zoneidx);
+extern bool compaction_allow_unevictable(void);
 
 #else
 static inline void reset_isolation_suitable(pg_data_t *pgdat)
@@ -131,6 +132,11 @@ static inline void wakeup_kcompactd(pg_data_t *pgdat,
 {
 }
 
+static inline bool compaction_allow_unevictable(void)
+{
+	return true;
+}
+
 #endif /* CONFIG_COMPACTION */
 
 struct node;
mm/compaction.c +7 -2
index 007d5e00a8ae..a10acb273454 100644
@@ -1341,6 +1341,7 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn,
 							unsigned long end_pfn)
 {
 	unsigned long pfn, block_start_pfn, block_end_pfn;
+	isolate_mode_t mode = cc->allow_unevictable ? ISOLATE_UNEVICTABLE : 0;
 	int ret = 0;
 
 	/* Scan block by block. First and last block may be incomplete */
@@ -1360,8 +1361,7 @@ isolate_migratepages_range(struct compact_control *cc, unsigned long start_pfn,
 					block_end_pfn, cc->zone))
 			continue;
 
-		ret = isolate_migratepages_block(cc, pfn, block_end_pfn,
-						 ISOLATE_UNEVICTABLE);
+		ret = isolate_migratepages_block(cc, pfn, block_end_pfn, mode);
 
 		if (ret)
 			break;
@@ -1902,6 +1902,11 @@ typedef enum {
  * compactable pages.
  */
 static int sysctl_compact_unevictable_allowed __read_mostly = CONFIG_COMPACT_UNEVICTABLE_DEFAULT;
+
+bool compaction_allow_unevictable(void)
+{
+	return sysctl_compact_unevictable_allowed;
+}
 /*
  * Tunable for proactive compaction. It determines how
  * aggressively the kernel should compact memory in the
mm/internal.h +1 -0
index 181e79f1d6a2..163f9d6b37f3 100644
@@ -1052,6 +1052,7 @@ struct compact_control {
 					 * ensure forward progress.
 					 */
 	bool alloc_contig;		/* alloc_contig_range allocation */
+	bool allow_unevictable;		/* Allow isolation of unevictable folios */
 };
 
 /*
mm/page_alloc.c +2 -0
index 81a9d4d1e6c0..1cf9d4a3b14c 100644
@@ -7118,6 +7118,8 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 		.ignore_skip_hint = true,
 		.no_set_skip_hint = true,
 		.alloc_contig = true,
+		.allow_unevictable = !!(alloc_flags & ACR_FLAGS_CMA) ||
+					     compaction_allow_unevictable(),
 	};
 	INIT_LIST_HEAD(&cc.migratepages);
 	enum pb_isolate_mode mode = (alloc_flags & ACR_FLAGS_CMA) ?
author
#18 lore raw
In reply to Wandun Chen
Quoted 6 lines · Wandun Chen

From: Wandun Chen <[email protected]>

vm.compact_unevictable_allowed=0 is meant to keep compaction from touching unevictable folios. In practice there are still two paths where it does not take effect. This series fixes them and adds a tracepoint to make such issues easier to diagnose in the future.

Gentle ping, any feedback on this series?

Sashiko found a potential compilation error in patch2, I'll address that along with any review comments in v2.

Best regards,
Wandun

Quoted 15 lines

Wandun Chen (3):
mm/compaction: skip isolate mlocked folios when
compact_unevictable_allowed=0
mm/compaction: add per-folio isolation tracepoint mm/compaction: respect compact_unevictable_allowed in alloc_contig
path

 include/linux/compaction.h        |  6 ++++++
 include/trace/events/compaction.h | 26 ++++++++++++++++++++++++++
 mm/compaction.c                   | 14 +++++++++++---
 mm/internal.h                     |  1 +
 mm/page_alloc.c                   |  2 ++
 5 files changed, 46 insertions(+), 3 deletions(-)
#19 lore raw
In reply to Wandun Chen

Errr... how does this relate to your other, non-RFC patch series [0] "mm: honour compact_unevictable_allowed in mlock and CMA paths"?

Why did you send this separately and why do you not mention the other series here or there?

And why is one RFC and the other not?

You really need to add more context like this, I spent quite a bit of time reviewing [0] and don't want to find out that actually you are abandoning that for this or something?...

Quoted 21 lines · Wandun Chen

From: Wandun Chen <[email protected]>

vm.compact_unevictable_allowed=0 is meant to keep compaction from touching unevictable folios. In practice there are still two paths where it does not take effect. This series fixes them and adds a tracepoint to make such issues easier to diagnose in the future.

Wandun Chen (3):
mm/compaction: skip isolate mlocked folios when
compact_unevictable_allowed=0
mm/compaction: add per-folio isolation tracepoint mm/compaction: respect compact_unevictable_allowed in alloc_contig
path

 include/linux/compaction.h        |  6 ++++++
 include/trace/events/compaction.h | 26 ++++++++++++++++++++++++++
 mm/compaction.c                   | 14 +++++++++++---
 mm/internal.h                     |  1 +
 mm/page_alloc.c                   |  2 ++
 5 files changed, 46 insertions(+), 3 deletions(-)
Quoted 3 lines

--
2.43.0

Thanks, Lorenzo

[0]:https://lore.kernel.org/all/[email protected]/

#20 lore raw
Quoted 11 lines · Lorenzo Stoakes

Errr... how does this relate to your other, non-RFC patch series [0] "mm: honour compact_unevictable_allowed in mlock and CMA paths"?

Why did you send this separately and why do you not mention the other series here or there?

And why is one RFC and the other not?

You really need to add more context like this, I spent quite a bit of time reviewing [0] and don't want to find out that actually you are abandoning that for this or something?...

Oh hang on... this is old, and [0] is new, my bad.

It just got pinged up my mail client because of recent replies :>)

Thanks, Lorenzo

Quoted 30 lines · Wandun Chen
Quoted 21 lines

From: Wandun Chen <[email protected]>

vm.compact_unevictable_allowed=0 is meant to keep compaction from touching unevictable folios. In practice there are still two paths where it does not take effect. This series fixes them and adds a tracepoint to make such issues easier to diagnose in the future.

Wandun Chen (3):
mm/compaction: skip isolate mlocked folios when
compact_unevictable_allowed=0
mm/compaction: add per-folio isolation tracepoint mm/compaction: respect compact_unevictable_allowed in alloc_contig
path

 include/linux/compaction.h        |  6 ++++++
 include/trace/events/compaction.h | 26 ++++++++++++++++++++++++++
 mm/compaction.c                   | 14 +++++++++++---
 mm/internal.h                     |  1 +
 mm/page_alloc.c                   |  2 ++
 5 files changed, 46 insertions(+), 3 deletions(-)
Quoted 3 lines

--
2.43.0

Thanks, Lorenzo

[0]:https://lore.kernel.org/all/[email protected]/