Skip to contentSkip to conversation

A minor flurry of selftest/mm fixes

PATCH In review John Hubbard 6 participants 39 messages Jun 6, 2023 – Dec 9, 2023

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

v3 cover A minor flurry of selftest/mm fixes

Hi,

Changes since v2 [1]:

* Added a new patch (sent separately earlier) at the end, to error out if "make headers" has not yet been run.

* Reworked and simplified the uffd movement patch. Now it only moves some uffd*() routines, not all, and doesn't have to touch the Makefile at all. This lighter touch also allowed me to drop the "move psize(), pshift() into vm_utils.c" entirely. I expect Peter Xu will be a little happier with this new approach.

* Fixed the commit description for the MADV_COLLAPSE patch.

* Added more Reviewed-by tags from David Hildenbrand and Peter Xu.

[1] https://lore.kernel.org/all/[email protected]/

John Hubbard (11):
selftests/mm: fix uffd-stress unused function warning selftests/mm: fix unused variable warnings in hugetlb-madvise.c,
migration.c
selftests/mm: fix "warning: expression which evaluates to zero..." in
mlock2-tests.c
selftests/mm: fix invocation of tests that are run via shell scripts selftests/mm: .gitignore: add mkdirty, va_high_addr_switch selftests/mm: fix two -Wformat-security warnings in uffd builds selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h selftests/mm: fix build failures due to missing MADV_COLLAPSE selftests/mm: move certain uffd*() routines from vm_util.c to
uffd-common.c
Documentation: kselftest: "make headers" is a prerequisite
selftests: error out if kernel header files are not yet built

 Documentation/dev-tools/kselftest.rst        |  1 +
 tools/testing/selftests/lib.mk               | 36 +++++++++++-
 tools/testing/selftests/mm/.gitignore        |  2 +
 tools/testing/selftests/mm/cow.c             |  7 ---
 tools/testing/selftests/mm/hugetlb-madvise.c |  8 ++-
 tools/testing/selftests/mm/khugepaged.c      | 10 ----
 tools/testing/selftests/mm/migration.c       |  5 +-
 tools/testing/selftests/mm/mlock2-tests.c    |  1 -
 tools/testing/selftests/mm/pkey-x86.h        |  2 +-
 tools/testing/selftests/mm/run_vmtests.sh    |  6 +-
 tools/testing/selftests/mm/uffd-common.c     | 59 ++++++++++++++++++++
 tools/testing/selftests/mm/uffd-common.h     |  5 ++
 tools/testing/selftests/mm/uffd-stress.c     | 10 ----
 tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++----
 tools/testing/selftests/mm/vm_util.c         | 59 --------------------
 tools/testing/selftests/mm/vm_util.h         | 14 +++--
 16 files changed, 130 insertions(+), 111 deletions(-)

base-commit: f8dba31b0a826e691949cd4fdfa5c30defaac8c5

Signature · 1 line
2.40.1
#1 lore raw

PATCH v3 1/11 selftests/mm: fix uffd-stress unused function warning

uffd_minor_feature() was unused. Remove it in order to fix the associated clang build warning.

---
 tools/testing/selftests/mm/uffd-stress.c | 10 ----------
 1 file changed, 10 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +0 -10
index f1ad9eef1c3a..995ff13e74c7 100644
@@ -88,16 +88,6 @@ static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)
 	}
 }
 
-static inline uint64_t uffd_minor_feature(void)
-{
-	if (test_type == TEST_HUGETLB && map_shared)
-		return UFFD_FEATURE_MINOR_HUGETLBFS;
-	else if (test_type == TEST_SHMEM)
-		return UFFD_FEATURE_MINOR_SHMEM;
-	else
-		return 0;
-}
-
 static void *locking_thread(void *arg)
 {
 	unsigned long cpu = (unsigned long) arg;
Signature · 1 line
2.40.1
Tested-by on v3 1/11
#2 lore raw

Re: [PATCH v3 01/11] selftests/mm: fix uffd-stress unused function warning

Quoted 6 lines · John Hubbard

uffd_minor_feature() was unused. Remove it in order to fix the associated clang build warning.

Quoted 25 lines
---
 tools/testing/selftests/mm/uffd-stress.c | 10 ----------
 1 file changed, 10 deletions(-)
tools/testing/selftests/mm/uffd-stress.c +0 -10
index f1ad9eef1c3a..995ff13e74c7 100644
@@ -88,16 +88,6 @@ static void uffd_stats_reset(struct uffd_args *args, unsigned long n_cpus)
 	}
 }
 
-static inline uint64_t uffd_minor_feature(void)
-{
-	if (test_type == TEST_HUGETLB && map_shared)
-		return UFFD_FEATURE_MINOR_HUGETLBFS;
-	else if (test_type == TEST_SHMEM)
-		return UFFD_FEATURE_MINOR_SHMEM;
-	else
-		return 0;
-}
-
 static void *locking_thread(void *arg)
 {
 	unsigned long cpu = (unsigned long) arg;
Signature · 2 lines
BR,
Muhammad Usama Anjum
#3 lore raw

PATCH v3 2/11 selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c

In reply to John Hubbard

Dummy variables are required in order to make these two (similar) routines work, so in both cases, declare the variables as volatile in order to avoid the clang compiler warning.

Furthermore, in order to ensure that each test actually does what is intended, add an asm volatile invocation (thanks to David Hildenbrand for the suggestion), with a clarifying comment so that it survives future maintenance.

---
 tools/testing/selftests/mm/hugetlb-madvise.c | 8 ++++++--
 tools/testing/selftests/mm/migration.c       | 5 ++++-
 2 files changed, 10 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/hugetlb-madvise.c +6 -2
index 28426e30d9bc..d55322df4b73 100644
@@ -65,11 +65,15 @@ void write_fault_pages(void *addr, unsigned long nr_pages)
 
 void read_fault_pages(void *addr, unsigned long nr_pages)
 {
-	unsigned long dummy = 0;
+	volatile unsigned long dummy = 0;
 	unsigned long i;
 
-	for (i = 0; i < nr_pages; i++)
+	for (i = 0; i < nr_pages; i++) {
 		dummy += *((unsigned long *)(addr + (i * huge_page_size)));
+
+		/* Prevent the compiler from optimizing out the entire loop: */
+		asm volatile("" : "+r" (dummy));
+	}
 }
 
 int main(int argc, char **argv)
tools/testing/selftests/mm/migration.c +4 -1
index 1cec8425e3ca..379581567f27 100644
@@ -95,12 +95,15 @@ int migrate(uint64_t *ptr, int n1, int n2)
 
 void *access_mem(void *ptr)
 {
-	uint64_t y = 0;
+	volatile uint64_t y = 0;
 	volatile uint64_t *x = ptr;
 
 	while (1) {
 		pthread_testcancel();
 		y += *x;
+
+		/* Prevent the compiler from optimizing out the writes to y: */
+		asm volatile("" : "+r" (y));
 	}
 
 	return NULL;
Signature · 1 line
2.40.1
Tested-by on v3 2/11
#4 lore raw

Re: [PATCH v3 02/11] selftests/mm: fix unused variable warnings in hugetlb-madvise.c, migration.c

Quoted 4 lines · John Hubbard

Dummy variables are required in order to make these two (similar) routines work, so in both cases, declare the variables as volatile in order to avoid the clang compiler warning.

Quoted 8 lines

Furthermore, in order to ensure that each test actually does what is intended, add an asm volatile invocation (thanks to David Hildenbrand for the suggestion), with a clarifying comment so that it survives future maintenance.

Quoted 48 lines
---
 tools/testing/selftests/mm/hugetlb-madvise.c | 8 ++++++--
 tools/testing/selftests/mm/migration.c       | 5 ++++-
 2 files changed, 10 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/hugetlb-madvise.c +6 -2
index 28426e30d9bc..d55322df4b73 100644
@@ -65,11 +65,15 @@ void write_fault_pages(void *addr, unsigned long nr_pages)
 
 void read_fault_pages(void *addr, unsigned long nr_pages)
 {
-	unsigned long dummy = 0;
+	volatile unsigned long dummy = 0;
 	unsigned long i;
 
-	for (i = 0; i < nr_pages; i++)
+	for (i = 0; i < nr_pages; i++) {
 		dummy += *((unsigned long *)(addr + (i * huge_page_size)));
+
+		/* Prevent the compiler from optimizing out the entire loop: */
+		asm volatile("" : "+r" (dummy));
+	}
 }
 
 int main(int argc, char **argv)
tools/testing/selftests/mm/migration.c +4 -1
index 1cec8425e3ca..379581567f27 100644
@@ -95,12 +95,15 @@ int migrate(uint64_t *ptr, int n1, int n2)
 
 void *access_mem(void *ptr)
 {
-	uint64_t y = 0;
+	volatile uint64_t y = 0;
 	volatile uint64_t *x = ptr;
 
 	while (1) {
 		pthread_testcancel();
 		y += *x;
+
+		/* Prevent the compiler from optimizing out the writes to y: */
+		asm volatile("" : "+r" (y));
 	}
 
 	return NULL;
Signature · 2 lines
BR,
Muhammad Usama Anjum
#5 lore raw

PATCH v3 3/11 selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c

In reply to John Hubbard

The stop variable is a char*, and the code was assigning a char value to it. This was generating a warning when compiling with clang.

However, as both David and Peter pointed out, stop is not even used after the problematic assignment to a char type. So just delete that line entirely.

---
 tools/testing/selftests/mm/mlock2-tests.c | 1 -
 1 file changed, 1 deletion(-)
tools/testing/selftests/mm/mlock2-tests.c +0 -1
index 11b2301f3aa3..80cddc0de206 100644
@@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)
 			printf("cannot parse /proc/self/maps\n");
 			goto out;
 		}
-		stop = '\0';
 
 		sscanf(line, "%lx", &start);
 		sscanf(end_addr, "%lx", &end);
Signature · 1 line
2.40.1
Tested-by on v3 3/11
#6 lore raw

Re: [PATCH v3 03/11] selftests/mm: fix "warning: expression which evaluates to zero..." in mlock2-tests.c

Quoted 10 lines · John Hubbard

The stop variable is a char*, and the code was assigning a char value to it. This was generating a warning when compiling with clang.

However, as both David and Peter pointed out, stop is not even used after the problematic assignment to a char type. So just delete that line entirely.

Quoted 16 lines
---
 tools/testing/selftests/mm/mlock2-tests.c | 1 -
 1 file changed, 1 deletion(-)
tools/testing/selftests/mm/mlock2-tests.c +0 -1
index 11b2301f3aa3..80cddc0de206 100644
@@ -50,7 +50,6 @@ static int get_vm_area(unsigned long addr, struct vm_boundaries *area)
 			printf("cannot parse /proc/self/maps\n");
 			goto out;
 		}
-		stop = '\0';
 
 		sscanf(line, "%lx", &start);
 		sscanf(end_addr, "%lx", &end);
Signature · 2 lines
BR,
Muhammad Usama Anjum
#7 lore raw

PATCH v3 4/11 selftests/mm: fix invocation of tests that are run via shell scripts

In reply to John Hubbard

We cannot depend upon git to reliably retain the executable bit on shell scripts, or so I was told several years ago while working on this same run_vmtests.sh script. And sure enough, things such as test_hmm.sh are lately failing to run, due to lacking execute permissions.

Fix this by explicitly adding "bash" to each of the shell script invocations. Leave fixing the overall approach to another day.

---
 tools/testing/selftests/mm/run_vmtests.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +3 -3
index 4893eb60d96d..8f81432e4bac 100644
@@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then
 	if [ "$ARCH" == "$ARCH_ARM64" ]; then
 		echo 6 > /proc/sys/vm/nr_hugepages
 	fi
-	CATEGORY="hugevm" run_test ./va_high_addr_switch.sh
+	CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh
 	if [ "$ARCH" == "$ARCH_ARM64" ]; then
 		echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages
 	fi
 fi # VADDR64
 
 # vmalloc stability smoke test
-CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke
+CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke
 
 CATEGORY="mremap" run_test ./mremap_dontunmap
 
-CATEGORY="hmm" run_test ./test_hmm.sh smoke
+CATEGORY="hmm" run_test bash ./test_hmm.sh smoke
 
 # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
 CATEGORY="madv_populate" run_test ./madv_populate
Signature · 1 line
2.40.1
Tested-by on v3 4/11
#8 lore raw

Re: [PATCH v3 04/11] selftests/mm: fix invocation of tests that are run via shell scripts

Quoted 11 lines · John Hubbard

We cannot depend upon git to reliably retain the executable bit on shell scripts, or so I was told several years ago while working on this same run_vmtests.sh script. And sure enough, things such as test_hmm.sh are lately failing to run, due to lacking execute permissions.

Fix this by explicitly adding "bash" to each of the shell script invocations. Leave fixing the overall approach to another day.

Quoted 30 lines
---
 tools/testing/selftests/mm/run_vmtests.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/mm/run_vmtests.sh +3 -3
index 4893eb60d96d..8f81432e4bac 100644
@@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then
 	if [ "$ARCH" == "$ARCH_ARM64" ]; then
 		echo 6 > /proc/sys/vm/nr_hugepages
 	fi
-	CATEGORY="hugevm" run_test ./va_high_addr_switch.sh
+	CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh
 	if [ "$ARCH" == "$ARCH_ARM64" ]; then
 		echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages
 	fi
 fi # VADDR64
 
 # vmalloc stability smoke test
-CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke
+CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke
 
 CATEGORY="mremap" run_test ./mremap_dontunmap
 
-CATEGORY="hmm" run_test ./test_hmm.sh smoke
+CATEGORY="hmm" run_test bash ./test_hmm.sh smoke
 
 # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
 CATEGORY="madv_populate" run_test ./madv_populate
Signature · 2 lines
BR,
Muhammad Usama Anjum
#9 lore raw

PATCH v3 5/11 selftests/mm: .gitignore: add mkdirty, va_high_addr_switch

In reply to John Hubbard

These new build products were left out of .gitignore, so add them now.

---
 tools/testing/selftests/mm/.gitignore | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/mm/.gitignore +2 -0
index 8917455f4f51..ab215303d8e9 100644
@@ -39,3 +39,5 @@ local_config.h
 local_config.mk
 ksm_functional_tests
 mdwe_test
+mkdirty
+va_high_addr_switch
\ No newline at end of file
Signature · 1 line
2.40.1
Tested-by on v3 5/11
#10 lore raw

Re: [PATCH v3 05/11] selftests/mm: .gitignore: add mkdirty, va_high_addr_switch

Quoted 1 line · John Hubbard

These new build products were left out of .gitignore, so add them now.

I'd added to the instructions that object files should be added to .gitignore. But still sometimes people forget.

Quoted 4 lines
Quoted 15 lines
---
 tools/testing/selftests/mm/.gitignore | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/mm/.gitignore +2 -0
index 8917455f4f51..ab215303d8e9 100644
@@ -39,3 +39,5 @@ local_config.h
 local_config.mk
 ksm_functional_tests
 mdwe_test
+mkdirty
+va_high_addr_switch
\ No newline at end of file
Signature · 2 lines
BR,
Muhammad Usama Anjum
#11 lore raw

PATCH v3 6/11 selftests/mm: fix two -Wformat-security warnings in uffd builds

In reply to John Hubbard

The uffd tests generate two compile time warnings from clang's -Wformat-security setting. These trigger at the call sites for uffd_test_start() and uffd_test_skip().

1) Fix the uffd_test_start() issue by removing the intermediate test_name variable (thanks to David Hildenbrand for showing how to do this).

2) Fix the uffd_test_skip() issue by observing that there is no need for a macro and a variable args approach, because all callers of uffd_test_skip() pass in a simple char* string, without any format specifiers. So just change uffd_test_skip() into a regular C function.

---
 tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)
tools/testing/selftests/mm/uffd-unit-tests.c +6 -10
index 269c86768a02..04d91f144d1c 100644
@@ -109,12 +109,11 @@ static void uffd_test_pass(void)
 		ksft_inc_fail_cnt();		\
 	} while (0)
 
-#define  uffd_test_skip(...)  do {		\
-		printf("skipped [reason: ");	\
-		printf(__VA_ARGS__);		\
-		printf("]\n");			\
-		ksft_inc_xskip_cnt();		\
-	} while (0)
+static void uffd_test_skip(const char *message)
+{
+	printf("skipped [reason: %s]\n", message);
+	ksft_inc_xskip_cnt();
+}
 
 /*
  * Returns 1 if specific userfaultfd supported, 0 otherwise.  Note, we'll
@@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])
 	uffd_test_case_t *test;
 	mem_type_t *mem_type;
 	uffd_test_args_t args;
-	char test_name[128];
 	const char *errmsg;
 	int has_uffd, opt;
 	int i, j;
@@ -1192,10 +1190,8 @@ int main(int argc, char *argv[])
 			mem_type = &mem_types[j];
 			if (!(test->mem_targets & mem_type->mem_flag))
 				continue;
-			snprintf(test_name, sizeof(test_name),
-				 "%s on %s", test->name, mem_type->name);
 
-			uffd_test_start(test_name);
+			uffd_test_start("%s on %s", test->name, mem_type->name);
 			if (!uffd_feature_supported(test)) {
 				uffd_test_skip("feature missing");
 				continue;
Signature · 1 line
2.40.1
Tested-by on v3 6/11
#12 lore raw

Re: [PATCH v3 06/11] selftests/mm: fix two -Wformat-security warnings in uffd builds

Quoted 8 lines · John Hubbard

The uffd tests generate two compile time warnings from clang's -Wformat-security setting. These trigger at the call sites for uffd_test_start() and uffd_test_skip().

1) Fix the uffd_test_start() issue by removing the intermediate test_name variable (thanks to David Hildenbrand for showing how to do this).

Quoted 8 lines

2) Fix the uffd_test_skip() issue by observing that there is no need for a macro and a variable args approach, because all callers of uffd_test_skip() pass in a simple char* string, without any format specifiers. So just change uffd_test_skip() into a regular C function.

Quoted 46 lines
---
 tools/testing/selftests/mm/uffd-unit-tests.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)
tools/testing/selftests/mm/uffd-unit-tests.c +6 -10
index 269c86768a02..04d91f144d1c 100644
@@ -109,12 +109,11 @@ static void uffd_test_pass(void)
 		ksft_inc_fail_cnt();		\
 	} while (0)
 
-#define  uffd_test_skip(...)  do {		\
-		printf("skipped [reason: ");	\
-		printf(__VA_ARGS__);		\
-		printf("]\n");			\
-		ksft_inc_xskip_cnt();		\
-	} while (0)
+static void uffd_test_skip(const char *message)
+{
+	printf("skipped [reason: %s]\n", message);
+	ksft_inc_xskip_cnt();
+}
 
 /*
  * Returns 1 if specific userfaultfd supported, 0 otherwise.  Note, we'll
@@ -1149,7 +1148,6 @@ int main(int argc, char *argv[])
 	uffd_test_case_t *test;
 	mem_type_t *mem_type;
 	uffd_test_args_t args;
-	char test_name[128];
 	const char *errmsg;
 	int has_uffd, opt;
 	int i, j;
@@ -1192,10 +1190,8 @@ int main(int argc, char *argv[])
 			mem_type = &mem_types[j];
 			if (!(test->mem_targets & mem_type->mem_flag))
 				continue;
-			snprintf(test_name, sizeof(test_name),
-				 "%s on %s", test->name, mem_type->name);
 
-			uffd_test_start(test_name);
+			uffd_test_start("%s on %s", test->name, mem_type->name);
 			if (!uffd_feature_supported(test)) {
 				uffd_test_skip("feature missing");
 				continue;
Signature · 2 lines
BR,
Muhammad Usama Anjum
#13 lore raw

PATCH v3 7/11 selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h

In reply to John Hubbard

This fixes a real bug, too, because xstate_size() was assuming that the stack variable xstate_size was initialized to zero. That's not guaranteed nor even especially likely.

---
 tools/testing/selftests/mm/pkey-x86.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/mm/pkey-x86.h +1 -1
index 72c14cd3ddc7..e32ae8a1cd99 100644
@@ -132,7 +132,7 @@ int pkey_reg_xstate_offset(void)
 	unsigned int ecx;
 	unsigned int edx;
 	int xstate_offset;
-	int xstate_size;
+	int xstate_size = 0;
 	unsigned long XSTATE_CPUID = 0xd;
 	int leaf;
 
Signature · 1 line
2.40.1
Tested-by on v3 7/11
#14 lore raw

Re: [PATCH v3 07/11] selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h

Quoted 7 lines · John Hubbard

This fixes a real bug, too, because xstate_size() was assuming that the stack variable xstate_size was initialized to zero. That's not guaranteed nor even especially likely.

Quoted 17 lines
---
 tools/testing/selftests/mm/pkey-x86.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/mm/pkey-x86.h +1 -1
index 72c14cd3ddc7..e32ae8a1cd99 100644
@@ -132,7 +132,7 @@ int pkey_reg_xstate_offset(void)
 	unsigned int ecx;
 	unsigned int edx;
 	int xstate_offset;
-	int xstate_size;
+	int xstate_size = 0;
 	unsigned long XSTATE_CPUID = 0xd;
 	int leaf;
 
Signature · 2 lines
BR,
Muhammad Usama Anjum
#15 lore raw

PATCH v3 8/11 selftests/mm: fix build failures due to missing MADV_COLLAPSE

In reply to John Hubbard

MADV_PAGEOUT, MADV_POPULATE_READ, MADV_COLLAPSE are conditionally defined as necessary. However, that was being done in .c files, and a new build failure came up that would have been automatically avoided had these been in a common header file.

So consolidate and move them all to vm_util.h, which fixes the build failure.

An alternative approach from Muhammad Usama Anjum was: rely on "make headers" being required, and include asm-generic/mman-common.h. This works in the sense that it builds, but it still generates warnings about duplicate MADV_* symbols, and the goal here is to get a fully clean (no warnings) build here.

---
 tools/testing/selftests/mm/cow.c        |  7 -------
 tools/testing/selftests/mm/khugepaged.c | 10 ----------
 tools/testing/selftests/mm/vm_util.h    | 10 ++++++++++
 3 files changed, 10 insertions(+), 17 deletions(-)
tools/testing/selftests/mm/cow.c +0 -7
index dc9d6fe86028..8882b05ec9c8 100644
@@ -30,13 +30,6 @@
 #include "../kselftest.h"
 #include "vm_util.h"
 
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
 static size_t pagesize;
 static int pagemap_fd;
 static size_t thpsize;
tools/testing/selftests/mm/khugepaged.c +0 -10
index 97adc0f34f9c..e88ee039d0eb 100644
@@ -22,16 +22,6 @@
 
 #include "vm_util.h"
 
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_POPULATE_READ
-#define MADV_POPULATE_READ 22
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
 #define BASE_ADDR ((void *)(1UL << 30))
 static unsigned long hpage_pmd_size;
 static unsigned long page_size;
tools/testing/selftests/mm/vm_util.h +10 -0
index b950bd16083a..07f39ed2efba 100644
@@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
 
 #define PAGEMAP_PRESENT(ent)	(((ent) & (1ull << 63)) != 0)
 #define PAGEMAP_PFN(ent)	((ent) & ((1ull << 55) - 1))
+
+#ifndef MADV_PAGEOUT
+#define MADV_PAGEOUT 21
+#endif
+#ifndef MADV_POPULATE_READ
+#define MADV_POPULATE_READ 22
+#endif
+#ifndef MADV_COLLAPSE
+#define MADV_COLLAPSE 25
+#endif
Signature · 1 line
2.40.1
Tested-by on v3 8/11
#16 lore raw

Re: [PATCH v3 08/11] selftests/mm: fix build failures due to missing MADV_COLLAPSE

Quoted 14 lines · John Hubbard

MADV_PAGEOUT, MADV_POPULATE_READ, MADV_COLLAPSE are conditionally defined as necessary. However, that was being done in .c files, and a new build failure came up that would have been automatically avoided had these been in a common header file.

So consolidate and move them all to vm_util.h, which fixes the build failure.

An alternative approach from Muhammad Usama Anjum was: rely on "make headers" being required, and include asm-generic/mman-common.h. This works in the sense that it builds, but it still generates warnings about duplicate MADV_* symbols, and the goal here is to get a fully clean (no warnings) build here.

Quoted 4 lines
Quoted 63 lines
---
 tools/testing/selftests/mm/cow.c        |  7 -------
 tools/testing/selftests/mm/khugepaged.c | 10 ----------
 tools/testing/selftests/mm/vm_util.h    | 10 ++++++++++
 3 files changed, 10 insertions(+), 17 deletions(-)
tools/testing/selftests/mm/cow.c +0 -7
index dc9d6fe86028..8882b05ec9c8 100644
@@ -30,13 +30,6 @@
 #include "../kselftest.h"
 #include "vm_util.h"
 
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
 static size_t pagesize;
 static int pagemap_fd;
 static size_t thpsize;
tools/testing/selftests/mm/khugepaged.c +0 -10
index 97adc0f34f9c..e88ee039d0eb 100644
@@ -22,16 +22,6 @@
 
 #include "vm_util.h"
 
-#ifndef MADV_PAGEOUT
-#define MADV_PAGEOUT 21
-#endif
-#ifndef MADV_POPULATE_READ
-#define MADV_POPULATE_READ 22
-#endif
-#ifndef MADV_COLLAPSE
-#define MADV_COLLAPSE 25
-#endif
-
 #define BASE_ADDR ((void *)(1UL << 30))
 static unsigned long hpage_pmd_size;
 static unsigned long page_size;
tools/testing/selftests/mm/vm_util.h +10 -0
index b950bd16083a..07f39ed2efba 100644
@@ -63,3 +63,13 @@ int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
 
 #define PAGEMAP_PRESENT(ent)	(((ent) & (1ull << 63)) != 0)
 #define PAGEMAP_PFN(ent)	((ent) & ((1ull << 55) - 1))
+
+#ifndef MADV_PAGEOUT
+#define MADV_PAGEOUT 21
+#endif
+#ifndef MADV_POPULATE_READ
+#define MADV_POPULATE_READ 22
+#endif
+#ifndef MADV_COLLAPSE
+#define MADV_COLLAPSE 25
+#endif
Signature · 2 lines
BR,
Muhammad Usama Anjum
#17 lore raw

PATCH v3 9/11 selftests/mm: move certain uffd*() routines from vm_util.c to uffd-common.c

In reply to John Hubbard

There are only three uffd*() routines that are used outside of the uffd selftests. Leave these in vm_util.c, where they are available to any mm selftest program:

uffd_register()
uffd_unregister()
uffd_register_with_ioctls().

A few other uffd*() routines, however, are only used by the uffd-focused tests found in uffd-stress.c and uffd-unit-tests.c. Move those routines into uffd-common.c.

---
 tools/testing/selftests/mm/uffd-common.c | 59 ++++++++++++++++++++++++
 tools/testing/selftests/mm/uffd-common.h |  5 ++
 tools/testing/selftests/mm/vm_util.c     | 59 ------------------------
 tools/testing/selftests/mm/vm_util.h     |  4 --
 4 files changed, 64 insertions(+), 63 deletions(-)
tools/testing/selftests/mm/uffd-common.c +59 -0
index 61c6250adf93..ba20d7504022 100644
@@ -616,3 +616,62 @@ int copy_page(int ufd, unsigned long offset, bool wp)
 {
 	return __copy_page(ufd, offset, false, wp);
 }
+
+int uffd_open_dev(unsigned int flags)
+{
+	int fd, uffd;
+
+	fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+	if (fd < 0)
+		return fd;
+	uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
+	close(fd);
+
+	return uffd;
+}
+
+int uffd_open_sys(unsigned int flags)
+{
+#ifdef __NR_userfaultfd
+	return syscall(__NR_userfaultfd, flags);
+#else
+	return -1;
+#endif
+}
+
+int uffd_open(unsigned int flags)
+{
+	int uffd = uffd_open_sys(flags);
+
+	if (uffd < 0)
+		uffd = uffd_open_dev(flags);
+
+	return uffd;
+}
+
+int uffd_get_features(uint64_t *features)
+{
+	struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
+	/*
+	 * This should by default work in most kernels; the feature list
+	 * will be the same no matter what we pass in here.
+	 */
+	int fd = uffd_open(UFFD_USER_MODE_ONLY);
+
+	if (fd < 0)
+		/* Maybe the kernel is older than user-only mode? */
+		fd = uffd_open(0);
+
+	if (fd < 0)
+		return fd;
+
+	if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
+		close(fd);
+		return -errno;
+	}
+
+	*features = uffdio_api.features;
+	close(fd);
+
+	return 0;
+}
tools/testing/selftests/mm/uffd-common.h +5 -0
index 6068f2346b86..197f5262fe0d 100644
@@ -110,6 +110,11 @@ int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
 int copy_page(int ufd, unsigned long offset, bool wp);
 void *uffd_poll_thread(void *arg);
 
+int uffd_open_dev(unsigned int flags);
+int uffd_open_sys(unsigned int flags);
+int uffd_open(unsigned int flags);
+int uffd_get_features(uint64_t *features);
+
 #define TEST_ANON	1
 #define TEST_HUGETLB	2
 #define TEST_SHMEM	3
tools/testing/selftests/mm/vm_util.c +0 -59

62 lines not on this page · raw message

tools/testing/selftests/mm/vm_util.h +0 -4
index 07f39ed2efba..c2d4ff798b91 100644
@@ -48,10 +48,6 @@ unsigned long default_huge_page_size(void);
 int uffd_register(int uffd, void *addr, uint64_t len,
 		  bool miss, bool wp, bool minor);
 int uffd_unregister(int uffd, void *addr, uint64_t len);
-int uffd_open_dev(unsigned int flags);
-int uffd_open_sys(unsigned int flags);
-int uffd_open(unsigned int flags);
-int uffd_get_features(uint64_t *features);
 int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
 			      bool miss, bool wp, bool minor, uint64_t *ioctls);
 
Signature · 1 line
2.40.1
Tested-by on v3 9/11
#18 lore raw

Re: [PATCH v3 09/11] selftests/mm: move certain uffd*() routines from vm_util.c to uffd-common.c

Quoted 8 lines · John Hubbard

There are only three uffd*() routines that are used outside of the uffd selftests. Leave these in vm_util.c, where they are available to any mm selftest program:

uffd_register()
uffd_unregister()
uffd_register_with_ioctls().

Quoted 7 lines

A few other uffd*() routines, however, are only used by the uffd-focused tests found in uffd-stress.c and uffd-unit-tests.c. Move those routines into uffd-common.c.

Quoted 172 lines
---
 tools/testing/selftests/mm/uffd-common.c | 59 ++++++++++++++++++++++++
 tools/testing/selftests/mm/uffd-common.h |  5 ++
 tools/testing/selftests/mm/vm_util.c     | 59 ------------------------
 tools/testing/selftests/mm/vm_util.h     |  4 --
 4 files changed, 64 insertions(+), 63 deletions(-)
tools/testing/selftests/mm/uffd-common.c +59 -0
index 61c6250adf93..ba20d7504022 100644
@@ -616,3 +616,62 @@ int copy_page(int ufd, unsigned long offset, bool wp)
 {
 	return __copy_page(ufd, offset, false, wp);
 }
+
+int uffd_open_dev(unsigned int flags)
+{
+	int fd, uffd;
+
+	fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
+	if (fd < 0)
+		return fd;
+	uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
+	close(fd);
+
+	return uffd;
+}
+
+int uffd_open_sys(unsigned int flags)
+{
+#ifdef __NR_userfaultfd
+	return syscall(__NR_userfaultfd, flags);
+#else
+	return -1;
+#endif
+}
+
+int uffd_open(unsigned int flags)
+{
+	int uffd = uffd_open_sys(flags);
+
+	if (uffd < 0)
+		uffd = uffd_open_dev(flags);
+
+	return uffd;
+}
+
+int uffd_get_features(uint64_t *features)
+{
+	struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
+	/*
+	 * This should by default work in most kernels; the feature list
+	 * will be the same no matter what we pass in here.
+	 */
+	int fd = uffd_open(UFFD_USER_MODE_ONLY);
+
+	if (fd < 0)
+		/* Maybe the kernel is older than user-only mode? */
+		fd = uffd_open(0);
+
+	if (fd < 0)
+		return fd;
+
+	if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
+		close(fd);
+		return -errno;
+	}
+
+	*features = uffdio_api.features;
+	close(fd);
+
+	return 0;
+}
tools/testing/selftests/mm/uffd-common.h +5 -0
index 6068f2346b86..197f5262fe0d 100644
@@ -110,6 +110,11 @@ int __copy_page(int ufd, unsigned long offset, bool retry, bool wp);
 int copy_page(int ufd, unsigned long offset, bool wp);
 void *uffd_poll_thread(void *arg);
 
+int uffd_open_dev(unsigned int flags);
+int uffd_open_sys(unsigned int flags);
+int uffd_open(unsigned int flags);
+int uffd_get_features(uint64_t *features);
+
 #define TEST_ANON	1
 #define TEST_HUGETLB	2
 #define TEST_SHMEM	3
tools/testing/selftests/mm/vm_util.c +0 -59
index 9b06a5034808..681277615839 100644
@@ -242,62 +242,3 @@ int uffd_unregister(int uffd, void *addr, uint64_t len)
 
 	return ret;
 }
-
-int uffd_open_dev(unsigned int flags)
-{
-	int fd, uffd;
-
-	fd = open("/dev/userfaultfd", O_RDWR | O_CLOEXEC);
-	if (fd < 0)
-		return fd;
-	uffd = ioctl(fd, USERFAULTFD_IOC_NEW, flags);
-	close(fd);
-
-	return uffd;
-}
-
-int uffd_open_sys(unsigned int flags)
-{
-#ifdef __NR_userfaultfd
-	return syscall(__NR_userfaultfd, flags);
-#else
-	return -1;
-#endif
-}
-
-int uffd_open(unsigned int flags)
-{
-	int uffd = uffd_open_sys(flags);
-
-	if (uffd < 0)
-		uffd = uffd_open_dev(flags);
-
-	return uffd;
-}
-
-int uffd_get_features(uint64_t *features)
-{
-	struct uffdio_api uffdio_api = { .api = UFFD_API, .features = 0 };
-	/*
-	 * This should by default work in most kernels; the feature list
-	 * will be the same no matter what we pass in here.
-	 */
-	int fd = uffd_open(UFFD_USER_MODE_ONLY);
-
-	if (fd < 0)
-		/* Maybe the kernel is older than user-only mode? */
-		fd = uffd_open(0);
-
-	if (fd < 0)
-		return fd;
-
-	if (ioctl(fd, UFFDIO_API, &uffdio_api)) {
-		close(fd);
-		return -errno;
-	}
-
-	*features = uffdio_api.features;
-	close(fd);
-
-	return 0;
-}
tools/testing/selftests/mm/vm_util.h +0 -4
index 07f39ed2efba..c2d4ff798b91 100644
@@ -48,10 +48,6 @@ unsigned long default_huge_page_size(void);
 int uffd_register(int uffd, void *addr, uint64_t len,
 		  bool miss, bool wp, bool minor);
 int uffd_unregister(int uffd, void *addr, uint64_t len);
-int uffd_open_dev(unsigned int flags);
-int uffd_open_sys(unsigned int flags);
-int uffd_open(unsigned int flags);
-int uffd_get_features(uint64_t *features);
 int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
 			      bool miss, bool wp, bool minor, uint64_t *ioctls);
 
Signature · 2 lines
BR,
Muhammad Usama Anjum
#19 lore raw

PATCH v3 10/11 Documentation: kselftest: "make headers" is a prerequisite

In reply to John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

However, that's not yet documented anywhere. So add it to Documentation/dev-tools/kselftest.rst .

[1] https://lore.kernel.org/all/[email protected]/

---
 Documentation/dev-tools/kselftest.rst | 1 +
 1 file changed, 1 insertion(+)
Documentation/dev-tools/kselftest.rst +1 -0
index 12b575b76b20..6e35d042199c 100644
@@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode)
 
 To build the tests::
 
+  $ make headers
   $ make -C tools/testing/selftests
 
 To run the tests::
Signature · 1 line
2.40.1
Reviewed-by on v3 10/11
#20 lore raw

Re: [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite

Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

However, that's not yet documented anywhere. So add it to Documentation/dev-tools/kselftest.rst .

Quoted 8 lines

[1] https://lore.kernel.org/all/[email protected]/

Quoted 16 lines
---
 Documentation/dev-tools/kselftest.rst | 1 +
 1 file changed, 1 insertion(+)
Documentation/dev-tools/kselftest.rst +1 -0
index 12b575b76b20..6e35d042199c 100644
@@ -36,6 +36,7 @@ Running the selftests (hotplug tests are run in limited mode)
 
 To build the tests::
 
+  $ make headers
   $ make -C tools/testing/selftests
 
 To run the tests::
Signature · 2 lines
BR,
Muhammad Usama Anjum
#21 lore raw

Re: [PATCH v3 10/11] Documentation: kselftest: "make headers" is a prerequisite

In reply to John Hubbard
Quoted 7 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

However, that's not yet documented anywhere. So add it to Documentation/dev-tools/kselftest.rst .

This is breaking the arm64 selftests, I've sent a revert:

https://lore.kernel.org/linux-kselftest/[email protected]/T/#u

(logs included in the above patch.)

#22 lore raw

PATCH v3 11/11 selftests: error out if kernel header files are not yet built

In reply to John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

[1] https://lore.kernel.org/all/[email protected]/

Cc David Hildenbrand <[email protected]>
Cc Peter Xu <[email protected]>
Cc Muhammad Usama Anjum <[email protected]>
Cc Jonathan Corbet <[email protected]>
---
 tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
 1 file changed, 33 insertions(+), 3 deletions(-)
tools/testing/selftests/lib.mk +33 -3
index 05400462c779..b8ea03b9a015 100644
@@ -44,10 +44,22 @@ endif
 selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
 top_srcdir = $(selfdir)/../../..
 
-ifeq ($(KHDR_INCLUDES),)
-KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
+ifneq ($(KBUILD_OUTPUT),)
+  # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
+  # expand a shell special character '~'. We use a somewhat tedious way here.
+  abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
+  $(if $(abs_objtree),, \
+    $(error failed to create output directory "$(KBUILD_OUTPUT)"))
+  # $(realpath ...) resolves symlinks
+  abs_objtree := $(realpath $(abs_objtree))
+  KHDR_DIR := ${abs_objtree}/usr/include
+else
+  abs_srctree := $(shell cd $(top_srcdir) && pwd)
+  KHDR_DIR := ${abs_srctree}/usr/include
 endif
 
+KHDR_INCLUDES := -isystem $(KHDR_DIR)
+
 # The following are built by lib.mk common compile rules.
 # TEST_CUSTOM_PROGS should be used by tests that require
 # custom build rule and prevent common build rule use.
@@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
 TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
 TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
 
-all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
+all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
+     $(TEST_GEN_FILES)
+
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                      \
+	if [ $$? -ne 0 ]; then                                                 \
+            RED='\033[1;31m';                                                  \
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1; \
+	fi
+
+.PHONY: kernel_header_files
 
 define RUN_TESTS
 	BASE_DIR="$(selfdir)";			\
Signature · 1 line
2.40.1
#23 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 61 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

[1] https://lore.kernel.org/all/[email protected]/

Cc David Hildenbrand <[email protected]>
Cc Peter Xu <[email protected]>
Cc Muhammad Usama Anjum <[email protected]>
Cc Jonathan Corbet <[email protected]>
---
 tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
 1 file changed, 33 insertions(+), 3 deletions(-)
tools/testing/selftests/lib.mk +21 -3
index 05400462c779..b8ea03b9a015 100644
@@ -44,10 +44,22 @@ endif
 selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
 top_srcdir = $(selfdir)/../../..
 
-ifeq ($(KHDR_INCLUDES),)
-KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
+ifneq ($(KBUILD_OUTPUT),)
+  # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
+  # expand a shell special character '~'. We use a somewhat tedious way here.
+  abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
+  $(if $(abs_objtree),, \
+    $(error failed to create output directory "$(KBUILD_OUTPUT)"))
+  # $(realpath ...) resolves symlinks
+  abs_objtree := $(realpath $(abs_objtree))
+  KHDR_DIR := ${abs_objtree}/usr/include
+else
+  abs_srctree := $(shell cd $(top_srcdir) && pwd)
+  KHDR_DIR := ${abs_srctree}/usr/include
 endif
 
+KHDR_INCLUDES := -isystem $(KHDR_DIR)
+
 # The following are built by lib.mk common compile rules.
 # TEST_CUSTOM_PROGS should be used by tests that require
 # custom build rule and prevent common build rule use.
@@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
 TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
 TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
 
-all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
+all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
+     $(TEST_GEN_FILES)
+
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                      \
+	if [ $$? -ne 0 ]; then                                                 \
+            RED='\033[1;31m';                                                  \

tools/testing/selftests/lib.mk TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS)) · line 89

Quoted 10 lines
tools/testing/selftests/lib.mk +10 -0
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1; \
+	fi

Thank you for adding this. This is outputting error for every selftest directory. We should try to make it even better by just aborting the Make-ing process the first time headers aren't detected. We can do this now or later, fine by me.

make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: *** [../lib.mk:77: kernel_header_files] Error 1 make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: *** [../lib.mk:77: kernel_header_files] Error 1 make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio' m

Complete error log file is attached.

Quoted 5 lines
tools/testing/selftests/lib.mk +2 -0
+
+.PHONY: kernel_header_files
 
 define RUN_TESTS
 	BASE_DIR="$(selfdir)";			\
Signature · 2 lines
BR,
Muhammad Usama Anjum

make: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/alsa'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/alsa' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/amd-pstate'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/amd-pstate' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/arm64' make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/arm64' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/breakpoints'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/breakpoints' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cachestat'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cachestat' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/capabilities'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/capabilities' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cgroup'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cgroup' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/clone3'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/clone3' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/core'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/core' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpufreq'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpufreq' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpu-hotplug'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/cpu-hotplug' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/damon'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/damon' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/dma-buf'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/dma-buf' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/s390x/uvdevice' make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/s390x/uvdevice' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/bonding'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/bonding' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/team'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/drivers/net/team' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/efivarfs'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/efivarfs' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/exec'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/exec' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/binderfs' make[1]: 'binderfs_test' is up to date.
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/binderfs' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/epoll'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/epoll' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/fat'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/filesystems/fat' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/firmware'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/firmware' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/fpu'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/fpu' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ftrace'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ftrace' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/futex' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/gpio' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/hid'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/hid' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/intel_pstate'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/intel_pstate' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/iommu'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/iommu' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ipc'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ipc' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ir'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ir' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kcmp'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kcmp' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kexec'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kexec' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kvm'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/kvm' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/landlock'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/landlock' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lib'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lib' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/livepatch'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/livepatch' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lkdtm'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/lkdtm' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/membarrier'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/membarrier' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memfd'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memfd' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memory-hotplug'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/memory-hotplug' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mincore'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mincore' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount_setattr'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mount_setattr' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/move_mount_set_group'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/move_mount_set_group' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mqueue'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mqueue' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nci'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nci' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/af_unix'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/af_unix' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/forwarding'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/forwarding' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/hsr'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/hsr' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/mptcp'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/mptcp' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/openvswitch'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/net/openvswitch' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/netfilter'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/netfilter' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nsfs'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/nsfs' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pidfd'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pidfd' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pid_namespace'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pid_namespace' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/powerpc'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/powerpc' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/prctl'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/prctl' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/proc'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/proc' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pstore'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/pstore' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ptrace'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/ptrace' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/openat2'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/openat2' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/resctrl'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/resctrl' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/riscv' make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/riscv' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rlimits'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rlimits' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rseq'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rseq' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rtc'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/rtc' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/seccomp'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/seccomp' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sgx'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sgx' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sigaltstack'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sigaltstack' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/size'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/size' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sparc64' make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sparc64' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/splice'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/splice' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/static_keys'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/static_keys' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sync'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sync' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/syscall_user_dispatch'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/syscall_user_dispatch' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sysctl'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/sysctl' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tc-testing'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tc-testing' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tdx'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tdx' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timens'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timens' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timers'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/timers' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tmpfs'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tmpfs' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tpm2'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/tpm2' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/user'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/user' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/vDSO'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/vDSO' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mm'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/mm' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/x86'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/x86' make[1]: Entering directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/zram'

-e error: missing kernel header files.
Please run this and try again:

cd /home/usama/repos/kernel/linux_mainline/tools/testing/selftests/../../.. make headers

make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/zram'
make: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests'

#24 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

On 6/6/23 00:38, Muhammad Usama Anjum wrote:
...

Quoted 10 lines
Quoted 10 lines
(quoted diff) +10 -0
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                      \
+	if [ $$? -ne 0 ]; then                                                 \
+            RED='\033[1;31m';                                                  \
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
Quoted 9 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1; \
+	fi

Thank you for adding this. This is outputting error for every selftest directory. We should try to make it even better by just aborting the Make-ing process the first time headers aren't detected. We can do this now or later, fine by me.

OK, I see. Yes, this can be improved by adding the same mechanism to the selftests/Makefile, that is in selftests/mm/Makefile.

I'd like to keep both, because as I mentioned earlier, mm folks like to run just that one Makefile, sometimes, and selftests/mm/Makefile is not invoking the top level Makefile. Rather, it includes lib.mk--which the top level Makefile does *not* include.

Arguably, using includes instead of recursive Make, would improve this
framework: reduce duplication such as the above. But that's a larger project and just food for thought at this point.

Anyway, this works nicely on my system, and I'll attach it as a patch also in case you want to try it out. What do you think of this:

tools/testing/selftests/Makefile +20 -1
index 90a62cf75008..bdca160063d8 100644
@@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
   abs_objtree := $(realpath $(abs_objtree))
   BUILD := $(abs_objtree)/kselftest
   KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
+  KHDR_DIR := ${abs_objtree}/usr/include
 else
   BUILD := $(CURDIR)
   abs_srctree := $(shell cd $(top_srcdir) && pwd)
   KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
+  KHDR_DIR := ${abs_srctree}/usr/include
   DEFAULT_INSTALL_HDR_PATH := 1
 endif
 
@@ -161,7 +163,7 @@ export KHDR_INCLUDES
 # all isn't the first target in the file.
 .DEFAULT_GOAL := all
 
-all:
+all: kernel_header_files
 	@ret=1;							\
 	for TARGET in $(TARGETS); do				\
 		BUILD_TARGET=$$BUILD/$$TARGET;			\
@@ -172,6 +174,23 @@ all:
 		ret=$$((ret * $$?));				\
 	done; exit $$ret;
 
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                          \
+	if [ $$? -ne 0 ]; then                                                     \
+            RED='\033[1;31m';                                                  \
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1;                                                                \
+	fi
+
+.PHONY: kernel_header_files
+
 run_tests: all
 	@for TARGET in $(TARGETS); do \
 		BUILD_TARGET=$$BUILD/$$TARGET;	\

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
tools/testing/selftests/Makefile +20 -1
index 90a62cf75008..bdca160063d8 100644
@@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
   abs_objtree := $(realpath $(abs_objtree))
   BUILD := $(abs_objtree)/kselftest
   KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
+  KHDR_DIR := ${abs_objtree}/usr/include
 else
   BUILD := $(CURDIR)
   abs_srctree := $(shell cd $(top_srcdir) && pwd)
   KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
+  KHDR_DIR := ${abs_srctree}/usr/include
   DEFAULT_INSTALL_HDR_PATH := 1
 endif
 
@@ -161,7 +163,7 @@ export KHDR_INCLUDES
 # all isn't the first target in the file.
 .DEFAULT_GOAL := all
 
-all:
+all: kernel_header_files
 	@ret=1;							\
 	for TARGET in $(TARGETS); do				\
 		BUILD_TARGET=$$BUILD/$$TARGET;			\
@@ -172,6 +174,23 @@ all:
 		ret=$$((ret * $$?));				\
 	done; exit $$ret;
 
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                          \
+	if [ $$? -ne 0 ]; then                                                     \
+            RED='\033[1;31m';                                                  \
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1;                                                                \
+	fi
+
+.PHONY: kernel_header_files
+
 run_tests: all
 	@for TARGET in $(TARGETS); do \
 		BUILD_TARGET=$$BUILD/$$TARGET;	\
#25 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 29 lines · John Hubbard

On 6/6/23 00:38, Muhammad Usama Anjum wrote:
...

Quoted 10 lines
Quoted 10 lines
tools/testing/selftests/Makefile +10 -0
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                      \
+	if [ $$? -ne 0 ]; then                                                 \
+            RED='\033[1;31m';                                                  \
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
Quoted 9 lines

tools/testing/selftests/Makefile all: · line 190

Quoted 4 lines
tools/testing/selftests/Makefile +4 -0
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1; \
+	fi

Thank you for adding this. This is outputting error for every selftest directory. We should try to make it even better by just aborting the Make-ing process the first time headers aren't detected. We can do this now or later, fine by me.

OK, I see. Yes, this can be improved by adding the same mechanism to the selftests/Makefile, that is in selftests/mm/Makefile.

I'd like to keep both, because as I mentioned earlier, mm folks like to run just that one Makefile, sometimes, and selftests/mm/Makefile is not invoking the top level Makefile. Rather, it includes lib.mk--which the top level Makefile does *not* include.

Quoted 6 lines

Arguably, using includes instead of recursive Make, would improve this
framework: reduce duplication such as the above. But that's a larger project and just food for thought at this point.

Anyway, this works nicely on my system, and I'll attach it as a patch also in case you want to try it out. What do you think of this:

Nice patch. Thanks. Lets add this patch as well. Please add the tag for this new patch:

Quoted 55 lines
tools/testing/selftests/Makefile +20 -1
index 90a62cf75008..bdca160063d8 100644
@@ -144,10 +144,12 @@ ifneq ($(KBUILD_OUTPUT),)
   abs_objtree := $(realpath $(abs_objtree))
   BUILD := $(abs_objtree)/kselftest
   KHDR_INCLUDES := -isystem ${abs_objtree}/usr/include
+  KHDR_DIR := ${abs_objtree}/usr/include
 else
   BUILD := $(CURDIR)
   abs_srctree := $(shell cd $(top_srcdir) && pwd)
   KHDR_INCLUDES := -isystem ${abs_srctree}/usr/include
+  KHDR_DIR := ${abs_srctree}/usr/include
   DEFAULT_INSTALL_HDR_PATH := 1
 endif
 
@@ -161,7 +163,7 @@ export KHDR_INCLUDES
 # all isn't the first target in the file.
 .DEFAULT_GOAL := all
 
-all:
+all: kernel_header_files
 	@ret=1;							\
 	for TARGET in $(TARGETS); do				\
 		BUILD_TARGET=$$BUILD/$$TARGET;			\
@@ -172,6 +174,23 @@ all:
 		ret=$$((ret * $$?));				\
 	done; exit $$ret;
 
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                          \
+	if [ $$? -ne 0 ]; then                                                     \
+            RED='\033[1;31m';                                                  \
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1;                                                                \
+	fi
+
+.PHONY: kernel_header_files
+
 run_tests: all
 	@for TARGET in $(TARGETS); do \
 		BUILD_TARGET=$$BUILD/$$TARGET;	\

thanks,

Signature · 2 lines
BR,
Muhammad Usama Anjum
Reviewed-by on v3 11/11
#26 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

In reply to John Hubbard
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

Quoted 8 lines

[1] https://lore.kernel.org/all/[email protected]/

Cc David Hildenbrand <[email protected]>
Cc Peter Xu <[email protected]>
Cc Muhammad Usama Anjum <[email protected]>
Cc Jonathan Corbet <[email protected]>
Quoted 60 lines
---
 tools/testing/selftests/lib.mk | 36 +++++++++++++++++++++++++++++++---
 1 file changed, 33 insertions(+), 3 deletions(-)
tools/testing/selftests/lib.mk +33 -3
index 05400462c779..b8ea03b9a015 100644
@@ -44,10 +44,22 @@ endif
 selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
 top_srcdir = $(selfdir)/../../..
 
-ifeq ($(KHDR_INCLUDES),)
-KHDR_INCLUDES := -isystem $(top_srcdir)/usr/include
+ifneq ($(KBUILD_OUTPUT),)
+  # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
+  # expand a shell special character '~'. We use a somewhat tedious way here.
+  abs_objtree := $(shell cd $(top_srcdir) && mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
+  $(if $(abs_objtree),, \
+    $(error failed to create output directory "$(KBUILD_OUTPUT)"))
+  # $(realpath ...) resolves symlinks
+  abs_objtree := $(realpath $(abs_objtree))
+  KHDR_DIR := ${abs_objtree}/usr/include
+else
+  abs_srctree := $(shell cd $(top_srcdir) && pwd)
+  KHDR_DIR := ${abs_srctree}/usr/include
 endif
 
+KHDR_INCLUDES := -isystem $(KHDR_DIR)
+
 # The following are built by lib.mk common compile rules.
 # TEST_CUSTOM_PROGS should be used by tests that require
 # custom build rule and prevent common build rule use.
@@ -58,7 +70,25 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
 TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
 TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
 
-all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
+all: kernel_header_files $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) \
+     $(TEST_GEN_FILES)
+
+kernel_header_files:
+	@ls $(KHDR_DIR)/linux/*.h >/dev/null 2>/dev/null;                      \
+	if [ $$? -ne 0 ]; then                                                 \
+            RED='\033[1;31m';                                                  \
+            NOCOLOR='\033[0m';                                                 \
+            echo;                                                              \
+            echo -e "$${RED}error$${NOCOLOR}: missing kernel header files.";   \
+            echo "Please run this and try again:";                             \
+            echo;                                                              \
+            echo "    cd $(top_srcdir)";                                       \
+            echo "    make headers";                                           \
+            echo;                                                              \
+	    exit 1; \
+	fi
+
+.PHONY: kernel_header_files
 
 define RUN_TESTS
 	BASE_DIR="$(selfdir)";			\
Signature · 2 lines
BR,
Muhammad Usama Anjum
#27 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

In reply to John Hubbard
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6

I've always done:

cd tools/testing/selftests/x86; make

and that has always worked

Now I can't bloody well build *any* selftest or risk not being able to do builds.

#28 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 21 lines · Peter Zijlstra
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

     make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6
Quoted 8 lines

I've always done:

cd tools/testing/selftests/x86; make

and that has always worked

Now I can't bloody well build *any* selftest or risk not being able to do builds.

This change landed in 6.5, no? And 6.6 was just released. Just curious why you notice that now.

Signature · 3 lines
Cheers,

David / dhildenb
#29 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 26 lines · David Hildenbrand
Quoted 25 lines · Peter Zijlstra
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

     make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6

I've always done:

cd tools/testing/selftests/x86; make

Quoted 7 lines
Quoted 4 lines

and that has always worked

Now I can't bloody well build *any* selftest or risk not being able to do builds.

This change landed in 6.5, no? And 6.6 was just released. Just curious why you notice that now.

Dunno, last time I edited the selftests and needed to recompile was a few weeks ago.

#30 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 29 lines · Peter Zijlstra
Quoted 28 lines · David Hildenbrand
Quoted 27 lines · Peter Zijlstra
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

      make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6

I've always done:

cd tools/testing/selftests/x86; make

and that has always worked

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Now I can't bloody well build *any* selftest or risk not being able to do builds.

This change landed in 6.5, no? And 6.6 was just released. Just curious why you notice that now.

Dunno, last time I edited the selftests and needed to recompile was a few weeks ago.

Okay. the question is if your workflow can be easily adjusted, or if we can improve that header handling as a whole.

The problem I had with this recently: just because we did a "make headers" once in a git tree doesn't mean that it is still up-to-date.

So once some selftest changes showed up that require newer headers, building the selftests again fails without a hint that another round of "make headers" would be required.

Signature · 3 lines
Cheers,

David / dhildenb
#31 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 42 lines · David Hildenbrand
Quoted 29 lines · Peter Zijlstra
Quoted 28 lines · David Hildenbrand
Quoted 27 lines · Peter Zijlstra
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

       make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6

I've always done:

cd tools/testing/selftests/x86; make

and that has always worked

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Now I can't bloody well build *any* selftest or risk not being able to do builds.

This change landed in 6.5, no? And 6.6 was just released. Just curious why you notice that now.

Dunno, last time I edited the selftests and needed to recompile was a few weeks ago.

Okay. the question is if your workflow can be easily adjusted, or if we can improve that header handling as a whole.

Quoted 6 lines

The problem I had with this recently: just because we did a "make headers" once in a git tree doesn't mean that it is still up-to-date.

So once some selftest changes showed up that require newer headers, building the selftests again fails without a hint that another round of "make headers" would be required.

To clarify: maybe some kind of a warning would be better, ideally that the headers might be outdated and that another "make headers" would be required in case there are any build errors.

Signature · 3 lines
Cheers,

David / dhildenb
#32 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

In reply to David Hildenbrand
Quoted 2 lines · David Hildenbrand

Okay. the question is if your workflow can be easily adjusted, or if we can improve that header handling as a whole.

So on IRC the following was suggested:

make O=defconfig-build headers ; make O=defconfig-build -C tools/testing/selftests/x86

But that makes absolutely no sense to me; because the headers and selftests are not .config dependent. Furthermore I don't want them in a kernel build dir.

Quoted 6 lines

The problem I had with this recently: just because we did a "make headers" once in a git tree doesn't mean that it is still up-to-date.

So once some selftest changes showed up that require newer headers, building the selftests again fails without a hint that another round of "make headers" would be required.

Yeah, so I've been adding #ifndef guards all over the place for decades and that just works. You need it in normal userspace too.

This super reliance on the very latestesetst headers is just a total pain.

#33 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

In reply to David Hildenbrand
Quoted 26 lines · David Hildenbrand
Quoted 25 lines · Peter Zijlstra
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

     make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6

I've always done:

cd tools/testing/selftests/x86; make

Quoted 7 lines
Quoted 4 lines

and that has always worked

Now I can't bloody well build *any* selftest or risk not being able to do builds.

This change landed in 6.5, no? And 6.6 was just released. Just curious why you notice that now.

And I hit it again (different box etc..)

Can we please get this garbage fixed already?

#34 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 32 lines · Peter Zijlstra
Quoted 31 lines · David Hildenbrand
Quoted 29 lines · Peter Zijlstra
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

      make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6

I've always done:

cd tools/testing/selftests/x86; make

and that has always worked

Now I can't bloody well build *any* selftest or risk not being able to do builds.

Quoted 6 lines
Quoted 2 lines

This change landed in 6.5, no? And 6.6 was just released. Just curious why you notice that now.

And I hit it again (different box etc..)

Can we please get this garbage fixed already?

I'd suggest to either revert or turn into a warning.

@John?

Signature · 3 lines
Cheers,

David / dhildenb
#35 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 33 lines · David Hildenbrand
Quoted 32 lines · Peter Zijlstra
Quoted 31 lines · David Hildenbrand
Quoted 29 lines · Peter Zijlstra
Quoted 8 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

      make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

NAK NAK NAK

This now means I can no longer run selftests, I thank you very much! :-/

root@spr:/usr/src/linux-2.6# make O=defconfig-build/ -j64
make[1]: Entering directory '/usr/src/linux-2.6/defconfig-build'
***
*** The source tree is not clean, please run 'make mrproper'
*** in /usr/src/linux-2.6

I've always done:

    cd tools/testing/selftests/x86; make

and that has always worked

Now I can't bloody well build *any* selftest or risk not being able to do builds.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

This change landed in 6.5, no? And 6.6 was just released. Just curious why you notice that now.

And I hit it again (different box etc..)

Can we please get this garbage fixed already?

I'd suggest to either revert or turn into a warning.

That would put us back into a half-broken sort of situation, though... see below.

Quoted 3 lines

@John?

I don't have a strong opinion about how this should be done, and in fact I believed at the time that I was bringing the system into compliance with what everyone wanted here. :)

There seem to be two conflicting visions:

a) The way it was (much) earlier: use ifdefs and defines to get by without the latest kernel headers, or

b) Requiring recent kernel headers to build the various selftests.

Shuah, Peter, others: can we choose a direction please? Either way will work, and I personally don't care which one we choose.

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
#36 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 6 lines · John Hubbard

I don't have a strong opinion about how this should be done, and in fact I believed at the time that I was bringing the system into compliance with what everyone wanted here. :)

There seem to be two conflicting visions:

Quoted 7 lines

a) The way it was (much) earlier: use ifdefs and defines to get by without the latest kernel headers, or

b) Requiring recent kernel headers to build the various selftests.

Shuah, Peter, others: can we choose a direction please? Either way will work, and I personally don't care which one we choose.

So as David already argued, the current thing does not in fact help with b. You just have to install once and the error goes away, then carry that tree for a year and you're running old crap again.

My biggest beef with the whole thing is that I simply do not want to use 'make headers', it doesn't work for me.

I have a ton of output directories and I don't care to build tools into the output dirs, in fact some of them flat out refuse to work that way (bpf comes to mind).

#37 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

Quoted 10 lines · Peter Zijlstra

So as David already argued, the current thing does not in fact help with b. You just have to install once and the error goes away, then carry that tree for a year and you're running old crap again.

My biggest beef with the whole thing is that I simply do not want to use 'make headers', it doesn't work for me.

I have a ton of output directories and I don't care to build tools into the output dirs, in fact some of them flat out refuse to work that way (bpf comes to mind).

Going with that, then, I believe it is best to simply revert commit 9fc96c7c19df ("selftests: error out if kernel header files are not yet built"). And then follow up with a series of (many) changes to wean the various selftests off of the kernel headers.

I'll post the revert shortly.

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
maintainer
#38 lore raw

Re: [PATCH v3 11/11] selftests: error out if kernel header files are not yet built

In reply to John Hubbard

Hi John, Muhammad,

Quoted 9 lines · John Hubbard

As per a discussion with Muhammad Usama Anjum [1], the following is how one is supposed to build selftests:

    make headers && make -C tools/testing/selftests/mm

Change the selftest build system's lib.mk to fail out with a helpful message if that prerequisite "make headers" has not been done yet.

[1] https://lore.kernel.org/all/[email protected]/

could you, please, elaborate more on that one is supposed to build selftests with 'make headers'? Yes, Documentation/dev-tools/kselftest.rst mentions that because you might need headers but...

The common way how we test the kernel is to build the kernel, install it somewhere and run selftests on top. The sequence basically being "make rpm-pkg; rpm -ivh; cd tools/testing/selftest/livepatch/ in source tree; sudo make run_tests" (or a similar variation of the procedure). The point is that we want to test the running kernel with its respective environment installed in /lib/modules/uname -r/ (if needed). This way we can run newer selftests from the current mainline tree on older kernels among others.

The commit breaks the use case which worked for a long long time.

It also breaks what Marcos proposed for livepatch selftests in
https://lore.kernel.org/all/[email protected]/

I guess we can always work around it by letting subsystem selftests to override KHDR_DIR but I am not comfortable with the behaviour that your commit introduced in the first place to be honest.

Thank you,
Miroslav