Skip to contentSkip to conversation

selftests/vm: calculate variables in correct order

PATCH In review Joel Savitz 4 participants 6 messages Oct 28, 2022 – Nov 17, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 selftests/vm: calculate variables in correct order

commit b5ba705c2608 ("selftests/vm: enable running select groups of tests") unintentionally reversed the ordering of some of the lines of run_vmtests.sh that calculate values based on system configuration. Importantly, $hpgsize_MB is determined from $hpgsize_KB, but this later value is not read from /proc/meminfo until later, causing userfaultfd tests to incorrectly fail since $half_ufd_size_MB will always be 0.

Switch these statements around into proper order to fix the invocation of the userfaultfd tests that use $half_ufd_size_MB.

---
 tools/testing/selftests/vm/run_vmtests.sh | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
tools/testing/selftests/vm/run_vmtests.sh +10 -10
index fff00bb77086..ce52e4f5ff21 100755
@@ -82,16 +82,6 @@ test_selected() {
 	fi
 }
 
-# Simple hugetlbfs tests have a hardcoded minimum requirement of
-# huge pages totaling 256MB (262144KB) in size.  The userfaultfd
-# hugetlb test requires a minimum of 2 * nr_cpus huge pages.  Take
-# both of these requirements into account and attempt to increase
-# number of huge pages available.
-nr_cpus=$(nproc)
-hpgsize_MB=$((hpgsize_KB / 1024))
-half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
-needmem_KB=$((half_ufd_size_MB * 2 * 1024))
-
 # get huge pagesize and freepages from /proc/meminfo
 while read -r name size unit; do
 	if [ "$name" = "HugePages_Free:" ]; then
@@ -102,6 +92,16 @@ while read -r name size unit; do
 	fi
 done < /proc/meminfo
 
+# Simple hugetlbfs tests have a hardcoded minimum requirement of
+# huge pages totaling 256MB (262144KB) in size.  The userfaultfd
+# hugetlb test requires a minimum of 2 * nr_cpus huge pages.  Take
+# both of these requirements into account and attempt to increase
+# number of huge pages available.
+nr_cpus=$(nproc)
+hpgsize_MB=$((hpgsize_KB / 1024))
+half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
+needmem_KB=$((half_ufd_size_MB * 2 * 1024))
+
 # set proper nr_hugepages
 if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
 	nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages)
Signature · 1 line
2.31.1
maintainer
#1 lore raw
Quoted 9 lines · On Fri, 28 Oct 2022 09:26:40 -0400 Joel Savitz <[email protected]> wrote:

commit b5ba705c2608 ("selftests/vm: enable running select groups of tests") unintentionally reversed the ordering of some of the lines of run_vmtests.sh that calculate values based on system configuration. Importantly, $hpgsize_MB is determined from $hpgsize_KB, but this later value is not read from /proc/meminfo until later, causing userfaultfd tests to incorrectly fail since $half_ufd_size_MB will always be 0.

Switch these statements around into proper order to fix the invocation of the userfaultfd tests that use $half_ufd_size_MB.

Does this fix address the failure in
https://lkml.kernel.org/r/[email protected]?

Thanks.

#2 lore raw
Quoted 10 lines · Andrew Morton
Quoted 1 line · On Fri, 28 Oct 2022 09:26:40 -0400 Joel Savitz <[email protected]> wrote:

commit b5ba705c2608 ("selftests/vm: enable running select groups of

tests")

Quoted 6 lines

unintentionally reversed the ordering of some of the lines of run_vmtests.sh that calculate values based on system configuration. Importantly, $hpgsize_MB is determined from $hpgsize_KB, but this later value is not read from /proc/meminfo until later, causing userfaultfd tests to incorrectly fail since $half_ufd_size_MB will always be 0.

Quoted 9 lines
Quoted 2 lines

Switch these statements around into proper order to fix the invocation of the userfaultfd tests that use $half_ufd_size_MB.

Does this fix address the failure in
https://lkml.kernel.org/r/[email protected]?

Thanks.

I have tried to reproduce this failure on a couple of different systems before and after the application of this commit but I haven't had any success in doing so. I suspect that there was some sort of hugepage configuration issue on the test system but I'd have to look into it more to be sure.

However, I noticed that on the mm-everything branch, the hugepage-mmap test fails:

# ./run_vmtests.sh -t "hugetlb"
running: ./hugepage-mmap
-----------------------
running ./hugepage-mmap
-----------------------
Open failed: No such file or directory
[FAIL]
...

It appears this is due to commit 0796c7b8be84 ("selftests/vm: drop mnt point for hugetlb in run_vmtests.sh")
as the test still replies on the ./huge mountpoint removed in that commit. The test passes before that patchset is applied.

Additionally, I just noticed an extraneous 'echo "running: $1"' line in run_test(), the effects of which are seen above, and I have just sent a patch to remove it.

Joel

#3 lore raw
Quoted 34 lines · Joel Savitz
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Fri, 28 Oct 2022 09:26:40 -0400 Joel Savitz <[email protected]> wrote:

commit b5ba705c2608 ("selftests/vm: enable running select groups of tests") unintentionally reversed the ordering of some of the lines of run_vmtests.sh that calculate values based on system configuration. Importantly, $hpgsize_MB is determined from $hpgsize_KB, but this later value is not read from /proc/meminfo until later, causing userfaultfd tests to incorrectly fail since $half_ufd_size_MB will always be 0.

Quoted 8 lines
Quoted 2 lines

Switch these statements around into proper order to fix the invocation of the userfaultfd tests that use $half_ufd_size_MB.

Does this fix address the failure in
https://lkml.kernel.org/r/[email protected]?

Thanks.

I have tried to reproduce this failure on a couple of different systems before and after the application of this commit but I haven't had any success in doing so. I suspect that there was some sort of hugepage configuration issue on the test system but I'd have to look into it more to be sure.

However, I noticed that on the mm-everything branch, the hugepage-mmap test fails:

# ./run_vmtests.sh -t "hugetlb"
running: ./hugepage-mmap
-----------------------
running ./hugepage-mmap
-----------------------
Open failed: No such file or directory
[FAIL]
...
Quoted 6 lines

It appears this is due to commit 0796c7b8be84 ("selftests/vm: drop mnt point for hugetlb in run_vmtests.sh") as the test still replies on the ./huge mountpoint removed in that commit. The test passes before that patchset is applied.

Additionally, I just noticed an extraneous 'echo "running: $1"' line in run_test(), the effects of which are seen above, and I have just sent a patch to remove it.

Joel

Resending this reply since it appears a bit of HTML slipped into the last reply and it got rejected by the lists.

#4 lore raw
In reply to Joel Savitz
Quoted 12 lines · Joel Savitz

However, I noticed that on the mm-everything branch, the hugepage-mmap test fails:

# ./run_vmtests.sh -t "hugetlb"
running: ./hugepage-mmap
-----------------------
running ./hugepage-mmap
-----------------------
Open failed: No such file or directory
[FAIL]
...
Quoted 4 lines

It appears this is due to commit 0796c7b8be84 ("selftests/vm: drop mnt point for hugetlb in run_vmtests.sh")
as the test still replies on the ./huge mountpoint removed in that commit. The test passes before that patchset is applied.

Oops, sorry I totally overlooked this hard-coded test case using the mntpoint.

Fix is simple though, which is attached.

Signature · 1 line
Peter Xu

From 71da2480d4bac0fc598e4d1f05f71aba8b980bc4 Mon Sep 17 00:00:00 2001
From: Peter Xu <[email protected]>
Date: Thu, 17 Nov 2022 16:29:15 -0500
Subject: [PATCH] selftests/vm: use memfd for hugepage-mmap test
Content-type: text/plain

This test was overlooked with a hard-coded mntpoint path in test when we're removing the hugetlb mntpoint in commit 0796c7b8be84. Fix it up so the test can keep running.

---
 tools/testing/selftests/vm/hugepage-mmap.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
tools/testing/selftests/vm/hugepage-mmap.c +4 -6
index 93f9e7b81331..955ef87f382c 100644
@@ -16,14 +16,13 @@
  * range.
  * Other architectures, such as ppc64, i386 or x86_64 are not so constrained.
  */
-
+#define _GNU_SOURCE
 #include <stdlib.h>
 #include <stdio.h>
 #include <unistd.h>
 #include <sys/mman.h>
 #include <fcntl.h>
 
-#define FILE_NAME "huge/hugepagefile"
 #define LENGTH (256UL*1024*1024)
 #define PROTECTION (PROT_READ | PROT_WRITE)
 
@@ -67,16 +66,16 @@ int main(void)
 	void *addr;
 	int fd, ret;
 
-	fd = open(FILE_NAME, O_CREAT | O_RDWR, 0755);
+	fd = memfd_create("hugepage-mmap", MFD_HUGETLB);
 	if (fd < 0) {
-		perror("Open failed");
+		perror("memfd_create() failed");
 		exit(1);
 	}
 
 	addr = mmap(ADDR, LENGTH, PROTECTION, FLAGS, fd, 0);
 	if (addr == MAP_FAILED) {
 		perror("mmap");
-		unlink(FILE_NAME);
+		close(fd);
 		exit(1);
 	}
 
@@ -87,7 +86,6 @@ int main(void)
 
 	munmap(addr, LENGTH);
 	close(fd);
-	unlink(FILE_NAME);
 
 	return ret;
 }
Signature · 1 line
2.37.3
Acked-by
#5 lore raw
Quoted 18 lines · Peter Xu
Quoted 12 lines · Joel Savitz

However, I noticed that on the mm-everything branch, the hugepage-mmap test fails:

# ./run_vmtests.sh -t "hugetlb"
running: ./hugepage-mmap
-----------------------
running ./hugepage-mmap
-----------------------
Open failed: No such file or directory
[FAIL]
...
Quoted 4 lines

It appears this is due to commit 0796c7b8be84 ("selftests/vm: drop mnt point for hugetlb in run_vmtests.sh")
as the test still replies on the ./huge mountpoint removed in that commit. The test passes before that patchset is applied.

Quoted 23 lines

Oops, sorry I totally overlooked this hard-coded test case using the mntpoint.

Fix is simple though, which is attached.

Signature · 17 lines
Peter Xu

From 71da2480d4bac0fc598e4d1f05f71aba8b980bc4 Mon Sep 17 00:00:00 2001
From: Peter Xu <[email protected]>
Date: Thu, 17 Nov 2022 16:29:15 -0500
Subject: [PATCH] selftests/vm: use memfd for hugepage-mmap test
Content-type: text/plain

This test was overlooked with a hard-coded mntpoint path in test when we're
removing the hugetlb mntpoint in commit 0796c7b8be84.  Fix it up so the
test can keep running.

Reported-by: Joel Savitz <[email protected]>
Signed-off-by: Peter Xu <[email protected]>
---
 tools/testing/selftests/vm/hugepage-mmap.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

Thanks Peter!
That is also something I noticed and was on my todo list.

Signature · 1 line
Mike Kravetz