Skip to contentSkip to conversation

Fix errors when try to build kvm selftests on specified output

PATCH Superseded Xiaoyao Li 2 participants 9 messages Mar 25, 2020 – Mar 26, 2020

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Xiaoyao Li

author
#0 lore raw

v2 cover Fix errors when try to build kvm selftests on specified output

I attempted to build KVM selftests on a specified dir, unfortunately
neither "make O=/path/to/mydir TARGETS=kvm" in tools/testing/selftests, nor
"make OUTPUT=/path/to/mydir" in tools/testing/selftests/kvm work.

This series aims to fix them.

Patch 1 fixes the issue that output directory is not exist.

Patch 2 and 3 are the preparation for kvm to get the right path of installed linux headers.

Patch 4 and 6 prepare the INSTALL_HDR_PATH to tell sub TARGET where the linux headers are installed.

Patch 5 fixes the issue that with OUTPUT specified, it still make the linux tree dirty.

I only test the sub TARGET of kvm.
In theory, it won't break other TARGET of selftests.

Changes in v2:
- fix the no directory issue in lib.mk
- make kvm fixes seperate patch
- Add the patch to fix linux src tree not clean issue

v1:
https://lore.kernel.org/kvm/[email protected]/

Xiaoyao Li (6):
selftests: Create directory when OUTPUT specified
selftests: kvm: Include lib.mk earlier
selftests: kvm: Use the default linux header path only when
INSTALL_HDR_PATH not defined
selftests: Create variable INSTALL_HDR_PATH if need to install linux

    headers to $(OUTPUT)/usr
  selftests: Generate build output of linux headers to
    $(OUTPUT)/linux-header-build

selftests: export INSTALL_HDR_PATH if using "O" to specify output dir

 tools/testing/selftests/Makefile     |  6 +++++-
 tools/testing/selftests/kvm/Makefile |  9 +++++----
 tools/testing/selftests/lib.mk       | 19 ++++++++++++++++++-
 3 files changed, 28 insertions(+), 6 deletions(-)
Signature · 1 line
2.20.1
author
#1 lore raw

PATCH v2 1/6 selftests: Create directory when OUTPUT specified

When OUTPUT specified, it may encounter no directory error if there is subdir in $TARGET for any of TEST_GEN_PROGS, TEST_GEN_PROGS_EXTENDED, or TEST_GEN_FILES.

This issue is found by running

make OUTPUT=/path/to/output

in tools/testing/selftests/kvm

---
 tools/testing/selftests/lib.mk | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/testing/selftests/lib.mk +10 -0
index 1c8a1963d03f..78fb00bdf5c4 100644
@@ -20,6 +20,16 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
 TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
 TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
 
+ifneq ($(TEST_GEN_PROGS),)
+$(shell mkdir -p $(sort $(dir $(TEST_GEN_PROGS))))
+endif
+ifneq ($(TEST_GEN_PROGS_EXTENDED),)
+$(shell mkdir -p $(sort $(dir $(TEST_GEN_PROGS_EXTENDED))))
+endif
+ifneq ($(TEST_GEN_FILES),)
+$(shell mkdir -p $(sort $(dir $(TEST_GEN_FILES))))
+endif
+
 ifdef KSFT_KHDR_INSTALL
 top_srcdir ?= ../../../..
 include $(top_srcdir)/scripts/subarch.include
Signature · 1 line
2.20.1
author
#2 lore raw

PATCH v2 2/6 selftests: kvm: Include lib.mk earlier

In reply to Xiaoyao Li

Include lib.mk a little bit earlier so that common lib.mk can setup environment earlier.

This is for the following patch to prepare INSTALL_HDR_PATH earlier.

---
 tools/testing/selftests/kvm/Makefile | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/Makefile +4 -3
index 67abc1dd50ee..504a5fb0fa71 100644
@@ -43,6 +43,10 @@ TEST_GEN_PROGS_s390x += kvm_create_max_vcpus
 TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(UNAME_M))
 LIBKVM += $(LIBKVM_$(UNAME_M))
 
+# After inclusion, $(OUTPUT) is defined and
+# $(TEST_GEN_PROGS) starts with $(OUTPUT)/
+include ../lib.mk
+
 INSTALL_HDR_PATH = $(top_srcdir)/usr
 LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/
 LINUX_TOOL_INCLUDE = $(top_srcdir)/tools/include
@@ -62,9 +66,6 @@ pgste-option = $(call try-run, echo 'int main() { return 0; }' | \
 
 LDFLAGS += -pthread $(no-pie-option) $(pgste-option)
 
-# After inclusion, $(OUTPUT) is defined and
-# $(TEST_GEN_PROGS) starts with $(OUTPUT)/
-include ../lib.mk
 
 STATIC_LIBS := $(OUTPUT)/libkvm.a
 LIBKVM_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM))
Signature · 1 line
2.20.1
author
#3 lore raw

PATCH v2 3/6 selftests: kvm: Use the default linux header path only when INSTALL_HDR_PATH not defined

In reply to Xiaoyao Li

This provides a way for lib.mk or parent Makefile to tell where the linux headers are installed.

---
 tools/testing/selftests/kvm/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile +1 -1
index 504a5fb0fa71..6f08cc512e15 100644
@@ -47,7 +47,7 @@ LIBKVM += $(LIBKVM_$(UNAME_M))
 # $(TEST_GEN_PROGS) starts with $(OUTPUT)/
 include ../lib.mk
 
-INSTALL_HDR_PATH = $(top_srcdir)/usr
+INSTALL_HDR_PATH ?= $(top_srcdir)/usr
 LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/
 LINUX_TOOL_INCLUDE = $(top_srcdir)/tools/include
 LINUX_TOOL_ARCH_INCLUDE = $(top_srcdir)/tools/arch/x86/include
Signature · 1 line
2.20.1
author
#4 lore raw

PATCH v2 4/6 selftests: Create variable INSTALL_HDR_PATH if need to install linux headers to $(OUTPUT)/usr

In reply to Xiaoyao Li

So sub TARGETs of selftest can know where the linux headers installed by checking INSTALL_HDR_PATH.

---
 tools/testing/selftests/lib.mk | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
tools/testing/selftests/lib.mk +7 -1
index 78fb00bdf5c4..b51ff93da4de 100644
@@ -35,6 +35,12 @@ top_srcdir ?= ../../../..
 include $(top_srcdir)/scripts/subarch.include
 ARCH		?= $(SUBARCH)
 
+ifndef KSFT_KHDR_INSTALL_DONE
+ifneq (1,$(DEFAULT_INSTALL_HDR_PATH))
+INSTALL_HDR_PATH := $(OUTPUT)/usr
+endif
+endif
+
 # set default goal to all, so make without a target runs all, even when
 # all isn't the first target in the file.
 .DEFAULT_GOAL := all
@@ -59,7 +65,7 @@ ifndef KSFT_KHDR_INSTALL_DONE
 ifeq (1,$(DEFAULT_INSTALL_HDR_PATH))
 	make --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
 else
-	make --no-builtin-rules INSTALL_HDR_PATH=$$OUTPUT/usr \
+	make --no-builtin-rules INSTALL_HDR_PATH=$(INSTALL_HDR_PATH) \
 		ARCH=$(ARCH) -C $(top_srcdir) headers_install
 endif
 endif
Signature · 1 line
2.20.1
author
#5 lore raw

PATCH v2 5/6 selftests: Generate build output of linux headers to $(OUTPUT)/linux-header-build

In reply to Xiaoyao Li

When build linux headers for selftests with OUTPUT specified, it makes linux src tree not clean.

Fix it by generating the output file of "headers_install" to a fixed dir in $(OUTPUT).

---
 tools/testing/selftests/lib.mk | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/lib.mk +1 -0
index b51ff93da4de..bb830e4817fc 100644
@@ -66,6 +66,7 @@ ifeq (1,$(DEFAULT_INSTALL_HDR_PATH))
 	make --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
 else
 	make --no-builtin-rules INSTALL_HDR_PATH=$(INSTALL_HDR_PATH) \
+		O=$(OUTPUT)/linux-header-build \
 		ARCH=$(ARCH) -C $(top_srcdir) headers_install
 endif
 endif
Signature · 1 line
2.20.1
author
#6 lore raw

PATCH v2 6/6 selftests: export INSTALL_HDR_PATH if using "O" to specify output dir

In reply to Xiaoyao Li

When build kvm selftests in tools/testing/selftests directory with

make O=/path/to/kselftests TARGETS=kvm

it fails building some kvm test binaries due to lack of header files.

Export INSTALL_HDR_PATH when "O" is specified, so that sub TARGET can get the right kernel headers with INSTALL_HDR_PATH.

---
 tools/testing/selftests/Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/testing/selftests/Makefile +5 -1
index be22dbe94a4c..f36bc6fd8086 100644
@@ -110,6 +110,10 @@ ARCH           ?= $(SUBARCH)
 export KSFT_KHDR_INSTALL_DONE := 1
 export BUILD
 
+ifneq (1,$(DEFAULT_INSTALL_HDR_PATH))
+export	INSTALL_HDR_PATH := $(BUILD)/usr
+endif
+
 # build and run gpio when output directory is the src dir.
 # gpio has dependency on tools/gpio and builds tools/gpio
 # objects in the src directory in all cases making the src
@@ -142,7 +146,7 @@ khdr:
 ifeq (1,$(DEFAULT_INSTALL_HDR_PATH))
 	$(MAKE) --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
 else
-	$(MAKE) --no-builtin-rules INSTALL_HDR_PATH=$$BUILD/usr \
+	$(MAKE) --no-builtin-rules INSTALL_HDR_PATH=$(INSTALL_HDR_PATH) \
 		ARCH=$(ARCH) -C $(top_srcdir) headers_install
 endif
 
Signature · 1 line
2.20.1
maintainer
#7 lore raw
In reply to Xiaoyao Li
Quoted 4 lines · Xiaoyao Li

I attempted to build KVM selftests on a specified dir, unfortunately
neither "make O=/path/to/mydir TARGETS=kvm" in tools/testing/selftests, nor
"make OUTPUT=/path/to/mydir" in tools/testing/selftests/kvm work.

Please elaborate on the problems you are seeing. I would like you to describe in detail the problems you are seeing and how you are fixing them in this patch series.

The problem you are fixing here is subdir structure not being supported for relocatable builds and the Makefile not being able to locate headers files. These are issues, however, these need to be fixed in the kvm Makefile.

Please look at arm64, android, futex tests as examples. lib.mk and main selftests Makefile allow for overrides for make targets. When a test has sub-dir structure and libraries, it is easier to handle these in the individual Makefile.

Please fix the problems you are seeing in kvm Makefile.

Quoted 2 lines

I only test the sub TARGET of kvm.
In theory, it won't break other TARGET of selftests.

When you change lib.mk which is a common infrastructure, theory doesn't help. Statements like this make me very reluctant to accept patches. :)

This is one reason why I asked Paolo to drop these patches.

thanks,
-- Shuah

author
#8 lore raw
Quoted 6 lines · shuah
Quoted 4 lines · Xiaoyao Li

I attempted to build KVM selftests on a specified dir, unfortunately neither    "make O=/path/to/mydir TARGETS=kvm" in tools/testing/selftests, nor "make OUTPUT=/path/to/mydir" in tools/testing/selftests/kvm work.

Quoted 8 lines

Please elaborate on the problems you are seeing. I would like you to describe in detail the problems you are seeing and how you are fixing them in this patch series.

The problem you are fixing here is subdir structure not being supported for relocatable builds and the Makefile not being able to locate headers files. These are issues, however, these need to be fixed in the kvm Makefile

Maybe I should have sent it as RFC.

OK, then I'll just resend patch 1 in v1 to fix the subdir not created issue for relocatable build, in kvm Makefile. Obviously lib.mk doesn't create it, and you don't allow me to touch lib.mk

About headers issue, since I'm not familiar with Makefile, I don't want to waste any more time on it and I decide to drop them.

I can add a "make mrproper" in my script to do the cleanup as a workaround.

And never make kvm selftests in parent dir with
make O=/somewhere TARGETS=kvm
to workaround the header not found issue.

Quoted 18 lines

Please look at arm64, android, futex tests as examples. lib.mk and main selftests Makefile allow for overrides for make targets. When a test has sub-dir structure and libraries, it is easier to handle these in the individual Makefile.

Please fix the problems you are seeing in kvm Makefile.

Quoted 2 lines

I only test the sub TARGET of kvm.
In theory, it won't break other TARGET of selftests.

When you change lib.mk which is a common infrastructure, theory doesn't help. Statements like this make me very reluctant to accept patches. :)

This is one reason why I asked Paolo to drop these patches.

thanks,
-- Shuah