Skip to contentSkip to conversation

selftests/bpf: Page out as late a possible in file_reader

PATCH Applied Jerome Marchand 6 participants 9 messages Apr 20 – Jun 2

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 selftests/bpf: Page out as late a possible in file_reader

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

A bit of instrumentation with mincore() shows that pages from that range were accessed several times before the program is run. In particular the call of file_reader__load() seems to fault all the range in.

Move the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those page pulled back in from under our feet.

---
 .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------
 1 file changed, 10 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/file_reader.c +10 -12
index 5cde32b35da44..48aae7ea0e4bb 100644
@@ -10,6 +10,7 @@
 
 const char *user_ptr = "hello world";
 char file_contents[256000];
+void *addr;
 
 void *get_executable_base_addr(void)
 {
@@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
 static int initialize_file_contents(void)
 {
 	int fd, page_sz = sysconf(_SC_PAGESIZE);
-	ssize_t n = 0, cur, off;
-	void *addr;
+	ssize_t n = 0, cur;
 
 	fd = open("/proc/self/exe", O_RDONLY);
 	if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
@@ -52,16 +52,6 @@ static int initialize_file_contents(void)
 	/* page-align base file address */
 	addr = (void *)((unsigned long)addr & ~(page_sz - 1));
 
-	/*
-	 * Page out range 0..512K, use 0..256K for positive tests and
-	 * 256K..512K for negative tests expecting page faults
-	 */
-	for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
-		if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
-			       "madvise pageout"))
-			return errno;
-	}
-
 	return 0;
 }
 
@@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
 	if (!ASSERT_OK(err, "file_reader__load"))
 		goto cleanup;
 
+	/*
+	 * Page out range 0..512K, use 0..256K for positive tests and
+	 * 256K..512K for negative tests expecting page faults
+	 */
+	if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, MADV_PAGEOUT),
+		       "madvise pageout"))
+		goto cleanup;
+
 	err = file_reader__attach(skel);
 	if (!ASSERT_OK(err, "file_reader__attach"))
 		goto cleanup;
Signature · 1 line
2.53.0
Acked-by on v1 1/1
#1 lore raw
Quoted 14 lines · Jerome Marchand

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

A bit of instrumentation with mincore() shows that pages from that range were accessed several times before the program is run. In particular the call of file_reader__load() seems to fault all the range in.

Quoted 6 lines

Move the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those page pulled back in from under our feet.

---

Thank you for the patch, the change looks good. Does it fail consistently on 4K page size?

Quoted 57 lines
  .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------
  1 file changed, 10 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/file_reader.c +10 -12
index 5cde32b35da44..48aae7ea0e4bb 100644
@@ -10,6 +10,7 @@
 
  const char *user_ptr = "hello world";
  char file_contents[256000];
+void *addr;
 
  void *get_executable_base_addr(void)
  {
@@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
  static int initialize_file_contents(void)
  {
  	int fd, page_sz = sysconf(_SC_PAGESIZE);
-	ssize_t n = 0, cur, off;
-	void *addr;
+	ssize_t n = 0, cur;
 
  	fd = open("/proc/self/exe", O_RDONLY);
  	if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
@@ -52,16 +52,6 @@ static int initialize_file_contents(void)
  	/* page-align base file address */
  	addr = (void *)((unsigned long)addr & ~(page_sz - 1));
 
-	/*
-	 * Page out range 0..512K, use 0..256K for positive tests and
-	 * 256K..512K for negative tests expecting page faults
-	 */
-	for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
-		if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
-			       "madvise pageout"))
-			return errno;
-	}
-
  	return 0;
  }
 
@@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
  	if (!ASSERT_OK(err, "file_reader__load"))
  		goto cleanup;
 
+	/*
+	 * Page out range 0..512K, use 0..256K for positive tests and
+	 * 256K..512K for negative tests expecting page faults
+	 */
+	if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, MADV_PAGEOUT),
+		       "madvise pageout"))
+		goto cleanup;
+
  	err = file_reader__attach(skel);
  	if (!ASSERT_OK(err, "file_reader__attach"))
  		goto cleanup;
#2 lore raw
Quoted 15 lines · Mykyta Yatsenko
Quoted 14 lines · Jerome Marchand

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

A bit of instrumentation with mincore() shows that pages from that range were accessed several times before the program is run. In particular the call of file_reader__load() seems to fault all the range in.

Quoted 8 lines
Quoted 6 lines

Move the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those page pulled back in from under our feet.

---

Thank you for the patch, the change looks good. Does it fail consistently on 4K page size?

It did when I ran the test manually. On our automated testing system, it failed intermittently.

Regards,
Jerome

Quoted 56 lines
Quoted 52 lines

  .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------

  1 file changed, 10 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/file_reader.c +10 -12
index 5cde32b35da44..48aae7ea0e4bb 100644
@@ -10,6 +10,7 @@
   const char *user_ptr = "hello world";
   char file_contents[256000];
+void *addr;
   void *get_executable_base_addr(void)
   {
@@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
   static int initialize_file_contents(void)
   {
       int fd, page_sz = sysconf(_SC_PAGESIZE);
-    ssize_t n = 0, cur, off;
-    void *addr;
+    ssize_t n = 0, cur;
       fd = open("/proc/self/exe", O_RDONLY);
       if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
@@ -52,16 +52,6 @@ static int initialize_file_contents(void)
       /* page-align base file address */
       addr = (void *)((unsigned long)addr & ~(page_sz - 1));
-    /*
-     * Page out range 0..512K, use 0..256K for positive tests and
-     * 256K..512K for negative tests expecting page faults
-     */
-    for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
-        if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
-                   "madvise pageout"))
-            return errno;
-    }
-
       return 0;
   }
@@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
       if (!ASSERT_OK(err, "file_reader__load"))
           goto cleanup;
+    /*
+     * Page out range 0..512K, use 0..256K for positive tests and
+     * 256K..512K for negative tests expecting page faults
+     */
+    if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, 
 MADV_PAGEOUT),
+               "madvise pageout"))
+        goto cleanup;
+
       err = file_reader__attach(skel);
       if (!ASSERT_OK(err, "file_reader__attach"))
           goto cleanup;
#3 lore raw
Quoted 20 lines · Jerome Marchand
Quoted 19 lines · Mykyta Yatsenko
Quoted 18 lines · Jerome Marchand

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

A bit of instrumentation with mincore() shows that pages from that range were accessed several times before the program is run. In particular the call of file_reader__load() seems to fault all the range in.

Move the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those page pulled back in from under our feet.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

---

Thank you for the patch, the change looks good. Does it fail consistently on 4K page size?

It did when I ran the test manually. On our automated testing system, it failed intermittently.

On my fork of BPF CI the test was failing[1] consistently, even when this commit is applied. However I wasn't able to reproduce locally on my laptop when I ran manually.

Any hints regarding where to look and how to reproduce?

Thanks,
Shung-Hsi

1: https://github.com/shunghsiyu/libbpf/actions/runs/26807198259/job/79027625188

...

#4 lore raw
Quoted 27 lines · Shung-Hsi Yu
Quoted 26 lines · Jerome Marchand
Quoted 24 lines · Mykyta Yatsenko
Quoted 14 lines · Jerome Marchand

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

A bit of instrumentation with mincore() shows that pages from that range were accessed several times before the program is run. In particular the call of file_reader__load() seems to fault all the range in.

Quoted 6 lines

Move the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those page pulled back in from under our feet.

---

Thank you for the patch, the change looks good. Does it fail consistently on 4K page size?

Quoted 8 lines
Quoted 2 lines

It did when I ran the test manually. On our automated testing system, it failed intermittently.

On my fork of BPF CI the test was failing[1] consistently, even when this commit is applied. However I wasn't able to reproduce locally on my laptop when I ran manually.

Any hints regarding where to look and how to reproduce?

To investigate the failure I witnessed I instrumented the test with some calls to mincore(). You can do that to see whether madvise(PAGEOUT) didn't successfully paged out those pages, or if they were paged back in by the MM subsystem. I'm afraid that this test isn't very robust: the MM subsystem is the one that ultimately decides which pages are in or out and it might not work in the way that this test assumes.

Thanks,
Jerome

Quoted 8 lines

Thanks,
Shung-Hsi

1: https://github.com/shunghsiyu/libbpf/actions/runs/26807198259/job/79027625188

...

#5 lore raw
Quoted 31 lines · Jerome Marchand
Quoted 30 lines · Shung-Hsi Yu
Quoted 28 lines · Jerome Marchand
Quoted 19 lines · Mykyta Yatsenko
Quoted 18 lines · Jerome Marchand

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

A bit of instrumentation with mincore() shows that pages from that range were accessed several times before the program is run. In particular the call of file_reader__load() seems to fault all the range in.

Move the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those page pulled back in from under our feet.

Quoted 5 lines
Quoted 2 lines

---

Thank you for the patch, the change looks good. Does it fail consistently on 4K page size?

It did when I ran the test manually. On our automated testing system, it failed intermittently.

Quoted 8 lines
Quoted 5 lines

On my fork of BPF CI the test was failing[1] consistently, even when this commit is applied. However I wasn't able to reproduce locally on my laptop when I ran manually.

Any hints regarding where to look and how to reproduce?

To investigate the failure I witnessed I instrumented the test with some calls to mincore(). You can do that to see whether madvise(PAGEOUT) didn't successfully paged out those pages, or if they were paged back in by the MM subsystem. I'm afraid that this test isn't very robust: the MM subsystem is the one that ultimately decides which pages are in or out and it might not work in the way that this test assumes.

thanks for the reports, this test is indeed flaky. I'll take a look if it's possible to fix it, or we should just delete it.

Quoted 12 lines

Thanks,
Jerome

Quoted 8 lines

Thanks,
Shung-Hsi

1: https://github.com/shunghsiyu/libbpf/actions/runs/26807198259/job/79027625188

...

#6 lore raw
In reply to Jerome Marchand
Sashiko AI review found 1 potential issue(s): - [Low] The newly promoted global variable `addr` is declared without the `static` keyword, polluting the global namespace. --

Sashiko AI review found 1 potential issue(s):
- [Low] The newly promoted global variable addr is declared without the static keyword, polluting the global namespace. --

commit e2c581de007abac2e6985b4f49f6c9786c3263b0
Author: Jerome Marchand <[email protected]>

selftests/bpf: Page out as late a possible in file_reader

The file_reader/on_open_expect_fault fails consistently on my system. This commit moves the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those pages pulled back in.

tools/testing/selftests/bpf/prog_tests/file_reader.c line 13

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/file_reader.c +1 -0
index 5cde32b35da44..48aae7ea0e4bb 100644
@@ -10,6 +10,7 @@
 
 const char *user_ptr = "hello world";
 char file_contents[256000];
+void *addr;

This isn't a bug, but BPF selftests link all prog_tests/*.c files together into a single test_progs executable.

Since modern compilers use -fno-common by default, could this un-prefixed global variable cause a multiple definition linker error if another test introduces a global variable with the same name?

Should addr be declared as static to safely restrict its linkage to file_reader.c?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Acked-by on v1 1/1
#7 lore raw
In reply to Jerome Marchand
Quoted 14 lines · Jerome Marchand

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

A bit of instrumentation with mincore() shows that pages from that range were accessed several times before the program is run. In particular the call of file_reader__load() seems to fault all the range in.

Quoted 5 lines

Move the call to madvise(MADV_PAGEOUT) to just before attaching the program to minimize the risk of having those page pulled back in from under our feet.

nit typo in the subject a -> as

jirka

Quoted 61 lines
---
 .../selftests/bpf/prog_tests/file_reader.c    | 22 +++++++++----------
 1 file changed, 10 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/file_reader.c +10 -12
index 5cde32b35da44..48aae7ea0e4bb 100644
@@ -10,6 +10,7 @@
 
 const char *user_ptr = "hello world";
 char file_contents[256000];
+void *addr;
 
 void *get_executable_base_addr(void)
 {
@@ -26,8 +27,7 @@ void *get_executable_base_addr(void)
 static int initialize_file_contents(void)
 {
 	int fd, page_sz = sysconf(_SC_PAGESIZE);
-	ssize_t n = 0, cur, off;
-	void *addr;
+	ssize_t n = 0, cur;
 
 	fd = open("/proc/self/exe", O_RDONLY);
 	if (!ASSERT_OK_FD(fd, "Open /proc/self/exe\n"))
@@ -52,16 +52,6 @@ static int initialize_file_contents(void)
 	/* page-align base file address */
 	addr = (void *)((unsigned long)addr & ~(page_sz - 1));
 
-	/*
-	 * Page out range 0..512K, use 0..256K for positive tests and
-	 * 256K..512K for negative tests expecting page faults
-	 */
-	for (off = 0; off < sizeof(file_contents) * 2; off += page_sz) {
-		if (!ASSERT_OK(madvise(addr + off, page_sz, MADV_PAGEOUT),
-			       "madvise pageout"))
-			return errno;
-	}
-
 	return 0;
 }
 
@@ -90,6 +80,14 @@ static void run_test(const char *prog_name)
 	if (!ASSERT_OK(err, "file_reader__load"))
 		goto cleanup;
 
+	/*
+	 * Page out range 0..512K, use 0..256K for positive tests and
+	 * 256K..512K for negative tests expecting page faults
+	 */
+	if (!ASSERT_OK(madvise(addr, sizeof(file_contents) * 2, MADV_PAGEOUT),
+		       "madvise pageout"))
+		goto cleanup;
+
 	err = file_reader__attach(skel);
 	if (!ASSERT_OK(err, "file_reader__attach"))
 		goto cleanup;
Signature · 1 line
2.53.0
#8 lore raw
In reply to Jerome Marchand
This patch was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Hello:

This patch was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Quoted 10 lines · On Mon, 20 Apr 2026 15:46:37 +0200 you wrote:

The file_reader/on_open_expect_fault fails consistently on my system. It expects a page fault on first dynptr read of some range the exe file of the current process because it has paged out that page range earlier. However a lot can happen to that range (which depending on the actual memory layout could contain text section, data section, sections )related to dynamic linking...) between the moment it was paged out and the moment the bpf program expected to hit a pagefault actually run.

[...]

Here is the summary with links:
- selftests/bpf: Page out as late a possible in file_reader

    https://git.kernel.org/bpf/bpf-next/c/a20f97791a78

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html