Skip to contentSkip to conversation

selftests: lib.mk: fix out-of-tree builds

PATCH In review Anders Roxell 4 participants 6 messages Jun 13, 2023 – Jun 14, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 selftests: lib.mk: fix out-of-tree builds

Since commit ("selftests: error out if kernel header files are not yet built") got merged, the kselftest build correctly because the KBUILD_OUTPUT isn't set when building out-of-tree and specifying 'O=' This is the error message that pops up.

make --silent --keep-going --jobs=32 O=/home/anders/.cache/tuxmake/builds/1482/build INSTALL_PATH=/home/anders/.cache/tuxmake/builds/1482/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- V=1 CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- kselftest-install make[3]: Entering directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa'

-e error: missing kernel header files.
Please run this and try again:

    cd /home/anders/src/kernel/next/tools/testing/selftests/../../..
    make headers

make[3]: Leaving directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa' make[3]: *** [../lib.mk:77: kernel_header_files] Error 1

Fixing the issue by assigning KBUILD_OUTPUT the same way how its done in kselftest's Makefile. By adding 'KBUILD_OUTPUT := $(O)' 'if $(origin O)' is set to 'command line'. This will set the the BUILD dir to KBUILD_OUTPUT/kselftest when doing out-of-tree builds which makes them in its own separete output directory.

---
 tools/testing/selftests/lib.mk | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/lib.mk +4 -0
index b8ea03b9a015..d17854285f2b 100644
@@ -44,6 +44,10 @@ endif
 selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
 top_srcdir = $(selfdir)/../../..
 
+ifeq ("$(origin O)", "command line")
+  KBUILD_OUTPUT := $(O)
+endif
+
 ifneq ($(KBUILD_OUTPUT),)
   # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
   # expand a shell special character '~'. We use a somewhat tedious way here.
Signature · 1 line
2.39.2
Tested-by on v1 1/1
#1 lore raw

This patch needs to go through mm tree as the commit which it fixes is in mm.

Quoted 17 lines · Anders Roxell

Since commit ("selftests: error out if kernel header files are not yet built") got merged, the kselftest build correctly because the KBUILD_OUTPUT isn't set when building out-of-tree and specifying 'O=' This is the error message that pops up.

make --silent --keep-going --jobs=32 O=/home/anders/.cache/tuxmake/builds/1482/build INSTALL_PATH=/home/anders/.cache/tuxmake/builds/1482/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- V=1 CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- kselftest-install make[3]: Entering directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa'

-e error: missing kernel header files.
Please run this and try again:

    cd /home/anders/src/kernel/next/tools/testing/selftests/../../..
    make headers

make[3]: Leaving directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa' make[3]: *** [../lib.mk:77: kernel_header_files] Error 1

Quoted 7 lines

Fixing the issue by assigning KBUILD_OUTPUT the same way how its done in kselftest's Makefile. By adding 'KBUILD_OUTPUT := $(O)' 'if $(origin O)' is set to 'command line'. This will set the the BUILD dir to KBUILD_OUTPUT/kselftest when doing out-of-tree builds which makes them in its own separete output directory.

Please add fixes tag. ("selftests: error out if kernel header files are not yet built")

Quoted 19 lines
---
 tools/testing/selftests/lib.mk | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/lib.mk +4 -0
index b8ea03b9a015..d17854285f2b 100644
@@ -44,6 +44,10 @@ endif
 selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
 top_srcdir = $(selfdir)/../../..
 
+ifeq ("$(origin O)", "command line")
+  KBUILD_OUTPUT := $(O)
+endif
+
 ifneq ($(KBUILD_OUTPUT),)
   # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
   # expand a shell special character '~'. We use a somewhat tedious way here.
Signature · 2 lines
BR,
Muhammad Usama Anjum
#2 lore raw
Quoted 26 lines · Muhammad Usama Anjum

This patch needs to go through mm tree as the commit which it fixes is in mm.

Quoted 23 lines · Anders Roxell

Since commit ("selftests: error out if kernel header files are not yet built") got merged, the kselftest build correctly because the KBUILD_OUTPUT isn't set when building out-of-tree and specifying 'O=' This is the error message that pops up.

make --silent --keep-going --jobs=32 O=/home/anders/.cache/tuxmake/builds/1482/build INSTALL_PATH=/home/anders/.cache/tuxmake/builds/1482/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- V=1 CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- kselftest-install make[3]: Entering directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa'

-e error: missing kernel header files.
Please run this and try again:

     cd /home/anders/src/kernel/next/tools/testing/selftests/../../..
     make headers

make[3]: Leaving directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa' make[3]: *** [../lib.mk:77: kernel_header_files] Error 1

Fixing the issue by assigning KBUILD_OUTPUT the same way how its done in kselftest's Makefile. By adding 'KBUILD_OUTPUT := $(O)' 'if $(origin O)' is set to 'command line'. This will set the the BUILD dir to KBUILD_OUTPUT/kselftest when doing out-of-tree builds which makes them in its own separete output directory.

Quoted 5 lines
Quoted 1 line

Please add fixes tag. ("selftests: error out if kernel header files are not yet built")

This should get squashed into the original patch unless it's already in mm-stable (which it isn't) and we have a stable commit id.

Signature · 3 lines
Cheers,

David / dhildenb
#3 lore raw
In reply to Anders Roxell
Quoted 33 lines · Anders Roxell

Since commit ("selftests: error out if kernel header files are not yet built") got merged, the kselftest build correctly because the KBUILD_OUTPUT isn't set when building out-of-tree and specifying 'O=' This is the error message that pops up.

make --silent --keep-going --jobs=32 O=/home/anders/.cache/tuxmake/builds/1482/build INSTALL_PATH=/home/anders/.cache/tuxmake/builds/1482/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- V=1 CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- kselftest-install make[3]: Entering directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa'

-e error: missing kernel header files.
Please run this and try again:

     cd /home/anders/src/kernel/next/tools/testing/selftests/../../..
     make headers

make[3]: Leaving directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa' make[3]: *** [../lib.mk:77: kernel_header_files] Error 1

Fixing the issue by assigning KBUILD_OUTPUT the same way how its done in kselftest's Makefile. By adding 'KBUILD_OUTPUT := $(O)' 'if $(origin O)' is set to 'command line'. This will set the the BUILD dir to KBUILD_OUTPUT/kselftest when doing out-of-tree builds which makes them in its own separete output directory.

---
  tools/testing/selftests/lib.mk | 4 ++++
  1 file changed, 4 insertions(+)
tools/testing/selftests/lib.mk +0 -0
index b8ea03b9a015..d17854285f2b 100644

tools/testing/selftests/lib.mk endif · line 50

Quoted 7 lines
tools/testing/selftests/lib.mk +4 -0
@@ -44,6 +44,10 @@ endif
  selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
  top_srcdir = $(selfdir)/../../..
 
+ifeq ("$(origin O)", "command line")
+  KBUILD_OUTPUT := $(O)
+endif
+

Thanks for fixing this up! This looks correct.

(It's too bad that we have all this duplication between the Makefile and lib.mk.)

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
Quoted 3 lines
tools/testing/selftests/lib.mk +0 -0
  ifneq ($(KBUILD_OUTPUT),)
    # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
    # expand a shell special character '~'. We use a somewhat tedious way here.
#4 lore raw
Quoted 45 lines · John Hubbard
Quoted 33 lines · Anders Roxell

Since commit ("selftests: error out if kernel header files are not yet built") got merged, the kselftest build correctly because the KBUILD_OUTPUT isn't set when building out-of-tree and specifying 'O=' This is the error message that pops up.

make --silent --keep-going --jobs=32 O=/home/anders/.cache/tuxmake/builds/1482/build INSTALL_PATH=/home/anders/.cache/tuxmake/builds/1482/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- V=1 CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- kselftest-install make[3]: Entering directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa'

-e [1;31merror [0m: missing kernel header files.
Please run this and try again:

     cd /home/anders/src/kernel/next/tools/testing/selftests/../../..
     make headers

make[3]: Leaving directory '/home/anders/src/kernel/next/tools/testing/selftests/alsa' make[3]: *** [../lib.mk:77: kernel_header_files] Error 1

Fixing the issue by assigning KBUILD_OUTPUT the same way how its done in kselftest's Makefile. By adding 'KBUILD_OUTPUT := $(O)' 'if $(origin O)' is set to 'command line'. This will set the the BUILD dir to KBUILD_OUTPUT/kselftest when doing out-of-tree builds which makes them in its own separete output directory.

---
  tools/testing/selftests/lib.mk | 4 ++++
  1 file changed, 4 insertions(+)
tools/testing/selftests/lib.mk +0 -0
index b8ea03b9a015..d17854285f2b 100644

tools/testing/selftests/lib.mk endif · line 50

Quoted 7 lines
tools/testing/selftests/lib.mk +4 -0
@@ -44,6 +44,10 @@ endif
  selfdir = $(realpath $(dir $(filter %/lib.mk,$(MAKEFILE_LIST))))
  top_srcdir = $(selfdir)/../../..
 
+ifeq ("$(origin O)", "command line")
+  KBUILD_OUTPUT := $(O)
+endif
+

Thanks for fixing this up! This looks correct.

Quoted 2 lines

(It's too bad that we have all this duplication between the Makefile and lib.mk.)

I agree, also the duplication of get_sys_includes in bpf, hid and net Makefile's Do you have any idea how we can remove the duplication?

Cheers,
Anders

Quoted 11 lines

thanks,
--
John Hubbard
NVIDIA

Quoted 3 lines
tools/testing/selftests/lib.mk +0 -0
  ifneq ($(KBUILD_OUTPUT),)
    # Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
    # expand a shell special character '~'. We use a somewhat tedious way here.
#5 lore raw
Quoted 6 lines · Anders Roxell
Quoted 2 lines

(It's too bad that we have all this duplication between the Makefile and lib.mk.)

I agree, also the duplication of get_sys_includes in bpf, hid and net Makefile's Do you have any idea how we can remove the duplication?

Well, since you asked... :)

Yes. Given that the goal is to support two similar, related activities:
a) make -C tools/testing/selftests
b) make -C tools/testing/selftests/<subtest>

, we could move common functionality into common_vars.mk and common_targets.mk, and include those from both the top level tools/testing/selftests/Makefile, and the subtest Makefiles.

This allows deduplication in a fairly safe sort of way, because it also simultaneously moves everything to a non-recursive Make design. That's especially nice, because then dependencies work reliably, and you also don't have to worry about propagating Make variables (via Make's export directive).

I haven't done this, because a few years ago I somehow got the vague impression that it would not be well-received. But maybe that's pessimistic and/or obsolete--times change, after all!

***************************************************************
Contents of tools/testing/selftests/Makefile:

include common_vars.mk

include alsa/alsa.mk
include amd-pstate/amd-pstate.mk
include arm64/arm64.mk
...etc, there are 100+ subdirs

include common_targets.mk

(This is instead of doing "TARGETS += xxx" in the Makefile. Including the subtest *.mk fragments allows each subtest to add features and quirks as necessary, while also enabling full dependency checking via a true non-recursive Make.)

***************************************************************
Contents of tools/testing/selftests/common_targets.mk:

all: kernel_headers

    ...

Other rules/targets here.

***************************************************************
Contents of tools/testing/selftests/common_vars.mk:

Common variables, some of which may be overridden by a Makefile.

***************************************************************

Using alsa as an example of a subtest:

***************************************************************
Contents of tools/testing/selftests/alsa/alsa.mk:
TARGETS += alsa

TEST* items

Plus: alsa-specific items, such as:
CFLAGS += $(shell pkg-config --cflags alsa)

***************************************************************
Contents of tools/testing/selftests/alsa/Makefile:

include ../common_vars.mk
include alsa.mk

...maybe very little else needs to go here.

include ../common_targets.mk

thanks,

Signature · 2 lines
John Hubbard
NVIDIA