Skip to contentSkip to conversation

selftests/nolibc: add minimal kernel config support

PATCH New Zhangjin Wu 3 participants 31 messages Jun 25, 2023 – Jul 18, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover selftests/nolibc: add minimal kernel config support

Willy, Thomas

We just sent the 'selftests/nolibc: allow run with minimal kernel config' series [1], Here is the 'minimal' kernel config support, with both of them, it is possible to run nolibc-test for all architectures with oneline command and in less than ~30 minutes - 1 hour (not fullly measured yet):

    // run with tiny config + qemu-system
    // Note: rv32 and loongarch require to download the bios at first
    $ time make run-tiny-all QUIET_RUN=1

    // run with default config + qemu-system
    $ time make run-default-all QUIET_RUN=1

    // run with qemu-user
    $ time make run-user-all QUIET_RUN=1

Besides the 'tinyconfig' suggestion from Thomas, this patch also merge the generic part of my local powerpc porting (the extconfig to add additional console support).

This is applied after the test report patchset [2] and the rv32 compile patchset [3], because all of them touched the same Makefile.

Even without the 'selftests/nolibc: allow run with minimal kernel config' series [1], all of the tests can pass except the /proc/self/net related ones (We haven't enable CONFIG_NET in this patchset), the chmod_net one will be removed by Thomas from this patchset [4] for the wrong chmodable attribute issue of /proc/self/net, the link_cross one can be simply fixed up by using another /proc/self interface (like /proc/self/cmdline), which will be covered in our revision of the [1] series.

Beside the core 'minimal' config support, some generic patch are added together to avoid patch conflicts.

* selftests/nolibc: add test for -include /path/to/nolibc.h

Add a test switch to allow run nolibc-test with nolibc.h

* selftests/nolibc: print result to the screen too

Let the run targets print results by default, allow disable by QUIET_RUN=1

* selftests/nolibc: allow use x86_64 toolchain for i386

Allow use x86_64 toolchains for i386

* selftests/nolibc: add menuconfig target for manual config

a new 'menuconfig' target added for development and debugging

* selftests/nolibc: add tinyconfig target

a new 'tinyconfig' compare to 'defconfig', smaller and faster, but not enough for boot and print, require following 'extconfig' target

* selftests/nolibc: allow customize extra kernel config options

a new 'extconfig' allows to add extra config options for 'defconfig' and 'tinyconfig'

* selftests/nolibc: add common extra config options selftests/nolibc: add power reset control support selftests/nolibc: add procfs, shmem and tmpfs

Add common extra configs, the 3rd one (procfs, shmem and tmpfs) can be completely reverted after [1] series, but as discuss with Thomas, procfs may be still a hard requirement.

* selftests/nolibc: add extra configs for i386 selftests/nolibc: add extra configs for x86_64 selftests/nolibc: add extra configs for arm64 selftests/nolibc: add extra configs for arm selftests/nolibc: add extra configs for mips selftests/nolibc: add extra configs for riscv32 selftests/nolibc: add extra configs for riscv64 selftests/nolibc: add extra configs for s390x selftests/nolibc: add extra configs for loongarch

Add architecture specific extra configs to let kernel boot and nolibc-test print. The rv32 added here is only for test, it should not be merged before the missing 64bit syscalls are added (still wait for the merging of the __sysret and -ENOSYS patches).

* selftests/nolibc: config default CROSS_COMPILE
selftests/nolibc: add run-tiny and run-default

both run-tiny and run-default are added to do config and run together, this easier test a log.

* selftests/nolibc: allow run tests on all targets selftests/nolibc: detect bios existing to avoid hang

Further allow do run-user, run-tiny and run-default for all architectures at once, the -all suffix is added to do so.

Since some generic patches are still in review, before sending the left rv32 patches, I'm will send more generic patches later, the coming one is arch-xxx.h cleanup, and then, the 32bit powerpc porting support.

For the compile speedup, the next step may be add architecture specific 'O' support, which may allow us rerun across architectures without mrproper, for a single architecture development, this 'minimal' config should be enough ;-)

Thanks.

Best regards,
Zhangjin
---
[1]: https://lore.kernel.org/lkml/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://lore.kernel.org/linux-riscv/[email protected]/
[4]: https://lore.kernel.org/lkml/[email protected]/

Zhangjin Wu (22):
selftests/nolibc: add test for -include /path/to/nolibc.h selftests/nolibc: print result to the screen too selftests/nolibc: allow use x86_64 toolchain for i386 selftests/nolibc: add menuconfig target for manual config selftests/nolibc: add tinyconfig target selftests/nolibc: allow customize extra kernel config options selftests/nolibc: add common extra config options selftests/nolibc: add power reset control support selftests/nolibc: add procfs, shmem and tmpfs selftests/nolibc: add extra configs for i386 selftests/nolibc: add extra configs for x86_64 selftests/nolibc: add extra configs for arm64 selftests/nolibc: add extra configs for arm selftests/nolibc: add extra configs for mips selftests/nolibc: add extra configs for riscv32 selftests/nolibc: add extra configs for riscv64 selftests/nolibc: add extra configs for s390x selftests/nolibc: add extra configs for loongarch selftests/nolibc: config default CROSS_COMPILE selftests/nolibc: add run-tiny and run-default selftests/nolibc: allow run tests on all targets selftests/nolibc: detect bios existing to avoid hang

 tools/testing/selftests/nolibc/Makefile | 125 ++++++++++++++++++++++--
 1 file changed, 119 insertions(+), 6 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/22 selftests/nolibc: add test for -include /path/to/nolibc.h

As the head comment of nolibc-test.c shows, it can be built in 3 ways:

  The program may be built in 3 ways:
    $(CC) -nostdlib -include /path/to/nolibc.h => NOLIBC already defined
    $(CC) -nostdlib -I/path/to/nolibc/sysroot  => _NOLIBC_* guards are present
    $(CC) with default libc                    => NOLIBC* never defined

Only last two of them are tested currently, let's allow test the first one too.

This may help to find issues about using nolibc.h to build programs. it derives from this change:

commit 3a8039e289a3 ("tools/nolibc: Fix build of stdio.h due to header ordering")

Usage:

// test with sysroot
$ make run-user SYSROOT=1

// test without sysroot, using nolibc.h directly $ make run-user SYSROOT=0

---
 tools/testing/selftests/nolibc/Makefile | 9 +++++++++
 1 file changed, 9 insertions(+)
tools/testing/selftests/nolibc/Makefile +9 -0
index 848884204a84..0b66b7a79cea 100644
@@ -102,6 +102,9 @@ REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++;print} /\[SKIPPED\][\
 		END{ printf("%d test(s) passed, %d skipped, %d failed.\n", p, s, f); \
 		printf("See all results in %s\n", ARGV[1]); }'
 
+# allow switch nolibc headers include method: 1 for sysroot, 0 for nolibc.h
+SYSROOT ?= 1
+
 help:
 	@echo "Supported targets under selftests/nolibc:"
 	@echo "  all          call the \"run\" target below"
@@ -139,9 +142,15 @@ sysroot/$(ARCH)/include:
 	$(Q)$(MAKE) -C ../../../include/nolibc ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone
 	$(Q)mv sysroot/sysroot sysroot/$(ARCH)
 
+ifeq ($(SYSROOT), 1)
 nolibc-test: nolibc-test.c sysroot/$(ARCH)/include
 	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \
 	  -nostdlib -static -Isysroot/$(ARCH)/include $< -lgcc
+else
+nolibc-test: nolibc-test.c
+	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \
+	  -nostdlib -static -include ../../../include/nolibc/nolibc.h $< -lgcc
+endif
 
 libc-test: nolibc-test.c
 	$(QUIET_CC)$(CC) -o $@ $<
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/22 selftests/nolibc: print result to the screen too

In reply to Zhangjin Wu

printing the running log to the screen helps users to learn what happens, especially to find potential issues immediately.

If want to silence a lot, simply pass a QUIET_RUN=1 is enough.

---
 tools/testing/selftests/nolibc/Makefile | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
tools/testing/selftests/nolibc/Makefile +11 -3
index 0b66b7a79cea..3300b5ab7de3 100644
@@ -98,6 +98,14 @@ CFLAGS  ?= -Os -fno-ident -fno-asynchronous-unwind-tables -std=c89 \
 		$(CFLAGS_$(ARCH)) $(CFLAGS_STACKPROTECTOR)
 LDFLAGS := -s
 
+# allow silence the running test log
+QUIET_RUN ?= 0
+ifeq ($(QUIET_RUN),0)
+LOG_OUTPUT = | tee "$(CURDIR)/run.out"
+else
+LOG_OUTPUT = > "$(CURDIR)/run.out"
+endif
+
 REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++;print} /\[SKIPPED\][\r]*$$/{s++} \
 		END{ printf("%d test(s) passed, %d skipped, %d failed.\n", p, s, f); \
 		printf("See all results in %s\n", ARGV[1]); }'
@@ -157,7 +165,7 @@ libc-test: nolibc-test.c
 
 # qemu user-land test
 run-user: nolibc-test
-	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
+	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test $(LOG_OUTPUT) || :
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 initramfs: nolibc-test
@@ -173,12 +181,12 @@ kernel: initramfs
 
 # run the tests after building the kernel
 run: kernel
-	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) $(LOG_OUTPUT)
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 # re-run the tests from an existing kernel
 rerun:
-	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) $(LOG_OUTPUT)
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 clean:
Signature · 1 line
2.25.1
#3 lore raw

PATCH 3/22 selftests/nolibc: allow use x86_64 toolchain for i386

In reply to Zhangjin Wu

Add extra '-m32 -Wl,-melf_i386' flags to tell x86_64 toolchains to generate i386 executable.

This allows to use the same x86_64 toolchain for i386 architecture.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 3300b5ab7de3..f8be0de0ce0e 100644
@@ -89,6 +89,7 @@ else
 Q=@
 endif
 
+CFLAGS_i386 = -m32 -Wl,-melf_i386
 CFLAGS_riscv32 = -march=rv32im -mabi=ilp32
 CFLAGS_s390 = -m64
 CFLAGS_mips = -EL
Signature · 1 line
2.25.1
#4 lore raw

PATCH 4/22 selftests/nolibc: add menuconfig target for manual config

In reply to Zhangjin Wu

The default DEFCONFIG_<ARCH> may not really work for all architectures, let's allow users to tune some configs with 'menuconfig'.

This is important when porting nolibc to a new architecture, it also allows to speed up nolibc 'run' target testing via manually disabling tons of unnecessary configs.

---
 tools/testing/selftests/nolibc/Makefile | 3 +++
 1 file changed, 3 insertions(+)
tools/testing/selftests/nolibc/Makefile +3 -0
index f8be0de0ce0e..10579396e60e 100644
@@ -177,6 +177,9 @@ initramfs: nolibc-test
 defconfig:
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
 
+menuconfig:
+	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) menuconfig
+
 kernel: initramfs
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs
 
Signature · 1 line
2.25.1
#5 lore raw

PATCH 5/22 selftests/nolibc: add tinyconfig target

In reply to Zhangjin Wu

The default DEFCONFIG_<ARCH> enables too many options, the kernel building with such options is very slow.

To speed up the 'run' target, in parallel with the 'defconfig' target, another 'tinyconfig' target is added to allow only enable necessary options to just let nolibc-test pass.

Note, this 'tinyconfig' target is not enough to let qemu really boot and print, the coming patches will add more options to let nolibc-test be able to run, print the results and let all tests pass.

---
 tools/testing/selftests/nolibc/Makefile | 3 +++
 1 file changed, 3 insertions(+)
tools/testing/selftests/nolibc/Makefile +3 -0
index 10579396e60e..5caf3e7023d7 100644
@@ -177,6 +177,9 @@ initramfs: nolibc-test
 defconfig:
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
 
+tinyconfig:
+	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare
+
 menuconfig:
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) menuconfig
 
Signature · 1 line
2.25.1
#6 lore raw

PATCH 6/22 selftests/nolibc: allow customize extra kernel config options

In reply to Zhangjin Wu

The default DEFCONFIG_<ARCH> may not really work for all architectures, tuning it with the 'menuconfig' target is good for development stage, but not enough for the default compile.

Since both of the 'defconfig' and 'tinyconfig' targets may not meet our requirement, just like the 'CONFIG_INITRAMFS_SOURCE' we added, some architectures may require to add extra configs, this adds a new 'extconfig' target to unify such requirement.

It allows to customize extra config options by architecture and then trigger 'allnoconfig' to let them take effect and with missing configs as disabled.

The old common 'CONFIG_INITRAMFS_SOURCE' setting from the 'kernel' target is moved to this target too.

It is safe enough to share this 'extconfig' between 'defconfig' and 'tinyconfig', because 'extconfig' is often a subset of 'defconfig' or an additions of 'defconfig' and 'tinyconfig'.

---
 tools/testing/selftests/nolibc/Makefile | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
tools/testing/selftests/nolibc/Makefile +15 -2
index 5caf3e7023d7..0b810681547b 100644
@@ -48,6 +48,15 @@ DEFCONFIG_s390       = defconfig
 DEFCONFIG_loongarch  = defconfig
 DEFCONFIG            = $(DEFCONFIG_$(ARCH))
 
+# extra kernel configs shared among architectures
+EXTCONFIG_COMMON     = --set-str CONFIG_INITRAMFS_SOURCE $(CURDIR)/initramfs
+
+# extra kernel configs by architecture
+EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
+
+# extra kernel configs, include common + architecture specific
+EXTCONFIG            = $(EXTCONFIG_ARCH) $(EXTCONFIG_COMMON)
+
 # optional tests to run (default = all)
 TEST =
 
@@ -183,8 +192,12 @@ tinyconfig:
 menuconfig:
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) menuconfig
 
-kernel: initramfs
-	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs
+extconfig:
+	$(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
+	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG=$(srctree)/.config allnoconfig
+
+kernel: initramfs extconfig
+	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME)
 
 # run the tests after building the kernel
 run: kernel
Signature · 1 line
2.25.1
#7 lore raw

PATCH 7/22 selftests/nolibc: add common extra config options

In reply to Zhangjin Wu

This adds config options required by all of the architectures.

* for initrd
-e BLK_DEV_INITRD --set-str CONFIG_INITRAMFS_SOURCE $(CURDIR)/initramfs

* for init executable
-e BINFMT_ELF

* for test result print
-e PRINTK -e TTY

* for panic reset instead of hang
--set-val PANIC_TIMEOUT 1

---
 tools/testing/selftests/nolibc/Makefile | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/selftests/nolibc/Makefile +4 -1
index 0b810681547b..147c2db49635 100644
@@ -49,7 +49,10 @@ DEFCONFIG_loongarch  = defconfig
 DEFCONFIG            = $(DEFCONFIG_$(ARCH))
 
 # extra kernel configs shared among architectures
-EXTCONFIG_COMMON     = --set-str CONFIG_INITRAMFS_SOURCE $(CURDIR)/initramfs
+EXTCONFIG_COMMON     = -e BLK_DEV_INITRD --set-str INITRAMFS_SOURCE $(CURDIR)/initramfs
+EXTCONFIG_COMMON    += --set-val PANIC_TIMEOUT 1
+EXTCONFIG_COMMON    += -e PRINTK -e TTY
+EXTCONFIG_COMMON    += -e BINFMT_ELF
 
 # extra kernel configs by architecture
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
Signature · 1 line
2.25.1
#8 lore raw

PATCH 8/22 selftests/nolibc: add power reset control support

In reply to Zhangjin Wu

Some boards require the power reset control options, they should have no side effect, enable them for all.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 147c2db49635..ae8519fb1b49 100644
@@ -53,6 +53,7 @@ EXTCONFIG_COMMON     = -e BLK_DEV_INITRD --set-str INITRAMFS_SOURCE $(CURDIR)/in
 EXTCONFIG_COMMON    += --set-val PANIC_TIMEOUT 1
 EXTCONFIG_COMMON    += -e PRINTK -e TTY
 EXTCONFIG_COMMON    += -e BINFMT_ELF
+EXTCONFIG_COMMON    += -e POWER_RESET -e POWER_RESET_SYSCON
 
 # extra kernel configs by architecture
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
Signature · 1 line
2.25.1
#9 lore raw

PATCH 9/22 selftests/nolibc: add procfs, shmem and tmpfs

In reply to Zhangjin Wu

Some test cases want to use procfs and tmpfs, let's enable them.

Because tmpfs depends on shmem, let's enable it too.

---
 tools/testing/selftests/nolibc/Makefile | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/nolibc/Makefile +2 -0
index ae8519fb1b49..1cddacbeb95e 100644
@@ -54,6 +54,8 @@ EXTCONFIG_COMMON    += --set-val PANIC_TIMEOUT 1
 EXTCONFIG_COMMON    += -e PRINTK -e TTY
 EXTCONFIG_COMMON    += -e BINFMT_ELF
 EXTCONFIG_COMMON    += -e POWER_RESET -e POWER_RESET_SYSCON
+EXTCONFIG_COMMON    += -e PROC_FS
+EXTCONFIG_COMMON    += -e SHMEM -e TMPFS
 
 # extra kernel configs by architecture
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
Signature · 1 line
2.25.1
#10 lore raw

PATCH 10/22 selftests/nolibc: add extra configs for i386

In reply to Zhangjin Wu

Enable CONFIG_COMPAT_32BIT_TIME for the poll test cases.

Add 8250 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 1cddacbeb95e..16bfb6f9644a 100644
@@ -58,6 +58,7 @@ EXTCONFIG_COMMON    += -e PROC_FS
 EXTCONFIG_COMMON    += -e SHMEM -e TMPFS
 
 # extra kernel configs by architecture
+EXTCONFIG_i386       = -e COMPAT_32BIT_TIME -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#11 lore raw

PATCH 11/22 selftests/nolibc: add extra configs for x86_64

In reply to Zhangjin Wu

Enable CONFIG_64BIT for x86_64 and x86.

Add 8250 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/nolibc/Makefile +2 -0
index 16bfb6f9644a..d45454f86ae1 100644
@@ -59,6 +59,8 @@ EXTCONFIG_COMMON    += -e SHMEM -e TMPFS
 
 # extra kernel configs by architecture
 EXTCONFIG_i386       = -e COMPAT_32BIT_TIME -e SERIAL_8250 -e SERIAL_8250_CONSOLE
+EXTCONFIG_x86_64     = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
+EXTCONFIG_x86        = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#12 lore raw

PATCH 12/22 selftests/nolibc: add extra configs for arm64

In reply to Zhangjin Wu

Add AMBA PL011 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index d45454f86ae1..6e59a694096d 100644
@@ -61,6 +61,7 @@ EXTCONFIG_COMMON    += -e SHMEM -e TMPFS
 EXTCONFIG_i386       = -e COMPAT_32BIT_TIME -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_x86_64     = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_x86        = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
+EXTCONFIG_arm64      = -e SERIAL_AMBA_PL011 -e SERIAL_AMBA_PL011_CONSOLE
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#13 lore raw

PATCH 13/22 selftests/nolibc: add extra configs for arm

In reply to Zhangjin Wu

Enable minimal configs for the arm virt board of qemu.

Enable CONFIG_COMPAT_32BIT_TIME for the poll test cases.

Add AMBA PL011 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 6e59a694096d..79983ddc1584 100644
@@ -62,6 +62,7 @@ EXTCONFIG_i386       = -e COMPAT_32BIT_TIME -e SERIAL_8250 -e SERIAL_8250_CONSOL
 EXTCONFIG_x86_64     = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_x86        = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_arm64      = -e SERIAL_AMBA_PL011 -e SERIAL_AMBA_PL011_CONSOLE
+EXTCONFIG_arm        = $(addprefix -e ,COMPAT_32BIT_TIME MMU ARCH_MULTIPLATFORM ARCH_MULTI_V7 ARM_PATCH_PHYS_VIRT ARCH_VIRT SERIAL_AMBA_PL011 SERIAL_AMBA_PL011_CONSOLE)
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#14 lore raw

PATCH 14/22 selftests/nolibc: add extra configs for mips

In reply to Zhangjin Wu

Enable minimal configs for the mips malta board of qemu.

Enable CONFIG_COMPAT_32BIT_TIME for the poll test cases.

Add 8250 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 79983ddc1584..234abf54cac5 100644
@@ -63,6 +63,7 @@ EXTCONFIG_x86_64     = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_x86        = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_arm64      = -e SERIAL_AMBA_PL011 -e SERIAL_AMBA_PL011_CONSOLE
 EXTCONFIG_arm        = $(addprefix -e ,COMPAT_32BIT_TIME MMU ARCH_MULTIPLATFORM ARCH_MULTI_V7 ARM_PATCH_PHYS_VIRT ARCH_VIRT SERIAL_AMBA_PL011 SERIAL_AMBA_PL011_CONSOLE)
+EXTCONFIG_mips       = $(addprefix -e ,COMPAT_32BIT_TIME MIPS_MALTA CPU_LITTLE_ENDIAN CPU_MIPS32_R2 SERIAL_8250 SERIAL_8250_CONSOLE)
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#15 lore raw

PATCH 15/22 selftests/nolibc: add extra configs for riscv32

In reply to Zhangjin Wu

Enable minimal configs for the riscv32 virt board of qemu.

Add 8250 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 234abf54cac5..3ba2c25eaa8e 100644
@@ -64,6 +64,7 @@ EXTCONFIG_x86        = -e 64BIT -e SERIAL_8250 -e SERIAL_8250_CONSOLE
 EXTCONFIG_arm64      = -e SERIAL_AMBA_PL011 -e SERIAL_AMBA_PL011_CONSOLE
 EXTCONFIG_arm        = $(addprefix -e ,COMPAT_32BIT_TIME MMU ARCH_MULTIPLATFORM ARCH_MULTI_V7 ARM_PATCH_PHYS_VIRT ARCH_VIRT SERIAL_AMBA_PL011 SERIAL_AMBA_PL011_CONSOLE)
 EXTCONFIG_mips       = $(addprefix -e ,COMPAT_32BIT_TIME MIPS_MALTA CPU_LITTLE_ENDIAN CPU_MIPS32_R2 SERIAL_8250 SERIAL_8250_CONSOLE)
+EXTCONFIG_riscv32    = $(addprefix -e ,MMU ARCH_RV32I NONPORTABLE SERIAL_8250 SERIAL_8250_CONSOLE SERIAL_OF_PLATFORM)
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#16 lore raw

PATCH 16/22 selftests/nolibc: add extra configs for riscv64

In reply to Zhangjin Wu

riscv implies riscv64, so, the same config options added for riscv and riscv64.

Add 8250 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/nolibc/Makefile +2 -0
index 3ba2c25eaa8e..8f9cb118cefe 100644
@@ -65,6 +65,8 @@ EXTCONFIG_arm64      = -e SERIAL_AMBA_PL011 -e SERIAL_AMBA_PL011_CONSOLE
 EXTCONFIG_arm        = $(addprefix -e ,COMPAT_32BIT_TIME MMU ARCH_MULTIPLATFORM ARCH_MULTI_V7 ARM_PATCH_PHYS_VIRT ARCH_VIRT SERIAL_AMBA_PL011 SERIAL_AMBA_PL011_CONSOLE)
 EXTCONFIG_mips       = $(addprefix -e ,COMPAT_32BIT_TIME MIPS_MALTA CPU_LITTLE_ENDIAN CPU_MIPS32_R2 SERIAL_8250 SERIAL_8250_CONSOLE)
 EXTCONFIG_riscv32    = $(addprefix -e ,MMU ARCH_RV32I NONPORTABLE SERIAL_8250 SERIAL_8250_CONSOLE SERIAL_OF_PLATFORM)
+EXTCONFIG_riscv64    = -e SERIAL_8250 -e SERIAL_8250_CONSOLE -e SERIAL_OF_PLATFORM
+EXTCONFIG_riscv      = -e SERIAL_8250 -e SERIAL_8250_CONSOLE -e SERIAL_OF_PLATFORM
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#17 lore raw

PATCH 17/22 selftests/nolibc: add extra configs for s390x

In reply to Zhangjin Wu

Add SCLP VT220 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 8f9cb118cefe..4d0de9e90d86 100644
@@ -67,6 +67,7 @@ EXTCONFIG_mips       = $(addprefix -e ,COMPAT_32BIT_TIME MIPS_MALTA CPU_LITTLE_E
 EXTCONFIG_riscv32    = $(addprefix -e ,MMU ARCH_RV32I NONPORTABLE SERIAL_8250 SERIAL_8250_CONSOLE SERIAL_OF_PLATFORM)
 EXTCONFIG_riscv64    = -e SERIAL_8250 -e SERIAL_8250_CONSOLE -e SERIAL_OF_PLATFORM
 EXTCONFIG_riscv      = -e SERIAL_8250 -e SERIAL_8250_CONSOLE -e SERIAL_OF_PLATFORM
+EXTCONFIG_s390       = -e SCLP_VT220_TTY -e SCLP_VT220_CONSOLE
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#18 lore raw

PATCH 18/22 selftests/nolibc: add extra configs for loongarch

In reply to Zhangjin Wu

Enable efi support for vmlinuz.efi boot.

Add 8250 console for test result print.

---
 tools/testing/selftests/nolibc/Makefile | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/nolibc/Makefile +1 -0
index 4d0de9e90d86..9092d209ebd0 100644
@@ -68,6 +68,7 @@ EXTCONFIG_riscv32    = $(addprefix -e ,MMU ARCH_RV32I NONPORTABLE SERIAL_8250 SE
 EXTCONFIG_riscv64    = -e SERIAL_8250 -e SERIAL_8250_CONSOLE -e SERIAL_OF_PLATFORM
 EXTCONFIG_riscv      = -e SERIAL_8250 -e SERIAL_8250_CONSOLE -e SERIAL_OF_PLATFORM
 EXTCONFIG_s390       = -e SCLP_VT220_TTY -e SCLP_VT220_CONSOLE
+EXTCONFIG_loongarch  = $(addprefix -e ,EFI_STUB EFI_ZBOOT SERIAL_8250 SERIAL_8250_CONSOLE SERIAL_8250_PNP)
 EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 
 # extra kernel configs, include common + architecture specific
Signature · 1 line
2.25.1
#19 lore raw

PATCH 19/22 selftests/nolibc: config default CROSS_COMPILE

In reply to Zhangjin Wu

This further simplify the 'make' commands, only require an ARCH now.

Almost all distributions provide toolchains for i386, x86_64, arm64, arm, mips, riscv32, riscv64 and s390x.

If can not find toolchains from the distribution repositories, we can download them from https://mirrors.edge.kernel.org/pub/tools/crosstool/
and then customize CROSS_COMPILE_<ARCH> for the testing.

---
 tools/testing/selftests/nolibc/Makefile | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
tools/testing/selftests/nolibc/Makefile +19 -0
index 9092d209ebd0..a847a2264fed 100644
@@ -74,6 +74,25 @@ EXTCONFIG_ARCH       = $(EXTCONFIG_$(ARCH))
 # extra kernel configs, include common + architecture specific
 EXTCONFIG            = $(EXTCONFIG_ARCH) $(EXTCONFIG_COMMON)
 
+# CROSS_COMPILE by architecture
+CROSS_COMPILE_i386      ?= x86_64-linux-gnu-
+CROSS_COMPILE_x86_64    ?= x86_64-linux-gnu-
+CROSS_COMPILE_x86       ?= x86_64-linux-gnu-
+CROSS_COMPILE_arm64     ?= aarch64-linux-gnu-
+CROSS_COMPILE_arm       ?= arm-linux-gnueabi-
+CROSS_COMPILE_mips      ?= mipsel-linux-gnu-
+CROSS_COMPILE_riscv32   ?= riscv64-linux-gnu-
+CROSS_COMPILE_riscv64   ?= riscv64-linux-gnu-
+CROSS_COMPILE_riscv     ?= riscv64-linux-gnu-
+CROSS_COMPILE_s390      ?= s390x-linux-gnu-
+CROSS_COMPILE_loongarch ?= loongarch64-linux-
+CROSS_COMPILE           ?= $(CROSS_COMPILE_$(ARCH))
+
+# Make sure CC has $(CROSS_COMPILE) prefix
+ifneq ($(CC),$(CROSS_COMPILE)$(CC))
+CC                      := $(CROSS_COMPILE)$(CC)
+endif
+
 # optional tests to run (default = all)
 TEST =
 
Signature · 1 line
2.25.1
#20 lore raw

PATCH 20/22 selftests/nolibc: add run-tiny and run-default

In reply to Zhangjin Wu

These two targets allow to do config and run together.

- run-tiny: do tinyconfig, extconfig and then run.
- run-default: do defconfig, extconfig and then run.

---
 tools/testing/selftests/nolibc/Makefile | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
tools/testing/selftests/nolibc/Makefile +11 -1
index a847a2264fed..3e7e51bd7082 100644
@@ -156,6 +156,14 @@ REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++;print} /\[SKIPPED\][\
 		END{ printf("%d test(s) passed, %d skipped, %d failed.\n", p, s, f); \
 		printf("See all results in %s\n", ARGV[1]); }'
 
+# allow run with a config target together: run-tiny = tinyconfig + run; run-default = defconfig + run
+ifeq (run-tiny, $(MAKECMDGOALS))
+AUTOCONFIG ?= tinyconfig
+endif
+ifeq (run-default, $(MAKECMDGOALS))
+AUTOCONFIG ?= defconfig
+endif
+
 # allow switch nolibc headers include method: 1 for sysroot, 0 for nolibc.h
 SYSROOT ?= 1
 
@@ -228,7 +236,7 @@ tinyconfig:
 menuconfig:
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) menuconfig
 
-extconfig:
+extconfig: $(AUTOCONFIG)
 	$(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG=$(srctree)/.config allnoconfig
 
@@ -240,6 +248,8 @@ run: kernel
 	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) $(LOG_OUTPUT)
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
+run-tiny run-default: run
+
 # re-run the tests from an existing kernel
 rerun:
 	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) $(LOG_OUTPUT)
Signature · 1 line
2.25.1
#21 lore raw

PATCH 21/22 selftests/nolibc: allow run tests on all targets

In reply to Zhangjin Wu

This lists all of the supported architectures and allow testing for all of them.

These targets are supported:

- run-user-all: testing run-user for all architectures
- run-all / run-default-all: testing run-default for all architectures
- run-tiny-all: testing run-tiny for all architectures

---
 tools/testing/selftests/nolibc/Makefile | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
tools/testing/selftests/nolibc/Makefile +16 -0
index 3e7e51bd7082..470923dc10e1 100644
@@ -164,6 +164,18 @@ ifeq (run-default, $(MAKECMDGOALS))
 AUTOCONFIG ?= defconfig
 endif
 
+# allow run tests on all architectures: run-user-all, run-all (=run-default-all), run-tiny-all
+ARCHS   ?= $(shell sed -ne 's/^DEFCONFIG_\([^ ]*\) .*/\1/p' $(CURDIR)/Makefile)
+GOALS   ?= run-user run-tiny run-default
+RUN_ALL ?= _t=$@; t=$${_t%-all}; [ "$$t" = "run" ] && t=run-default; \
+	   if echo $(GOALS) | grep -wq "$$t"; then \
+		for a in $(ARCHS); do echo "Testing $$t for $${a}:"; make $$t ARCH=$$a; cp $(CURDIR)/run.out $(CURDIR)/run-$$a.out; done; \
+		echo "\n\nTesting summary of $$t:\n"; \
+		for a in $(ARCHS); do echo $${a}:; echo; $(REPORT) $(CURDIR)/run-$$a.out; echo; done; \
+	   else \
+		echo "Ignoring unknown target: $$t, available targets: $(GOALS)"; \
+	   fi
+
 # allow switch nolibc headers include method: 1 for sysroot, 0 for nolibc.h
 SYSROOT ?= 1
 
@@ -255,6 +267,10 @@ rerun:
 	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) $(LOG_OUTPUT)
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
+# test for all architectures
+%-all:
+	$(Q)$(RUN_ALL)
+
 clean:
 	$(call QUIET_CLEAN, sysroot)
 	$(Q)rm -rf sysroot
Signature · 1 line
2.25.1
#22 lore raw

PATCH 22/22 selftests/nolibc: detect bios existing to avoid hang

In reply to Zhangjin Wu

Without a right -bios option, riscv32 and loongarch will hang during boot and therefore block the whole testing, this adds necessary detection.

Before testing, the required bios should be downloaded at first, for a future working qemu (without a manual -bios), we can simply clear the QEMU_BIOS_<ARCH> to stop the detection.

By default, the bios should be downloaded and put into tools/testing/selftests/nolibc/, otherwise, users should specify the path via QEMU_BIOS_<ARCH>.

Without this patch, it is not possible to directly run tests for all architectures, otherwise, we should pass our own 'ARCHS' and remove the unsupported ones explicitly, which is not convenient.

---
 tools/testing/selftests/nolibc/Makefile | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
tools/testing/selftests/nolibc/Makefile +16 -2
index 470923dc10e1..b529bb40444a 100644
@@ -110,6 +110,14 @@ QEMU_ARCH_s390       = s390x
 QEMU_ARCH_loongarch  = loongarch64
 QEMU_ARCH            = $(QEMU_ARCH_$(ARCH))
 
+# QEMU_BIOS: bios used by qemu
+# https://github.com/loongson/Firmware/raw/main/LoongArchVirtMachine/edk2-loongarch64-code.fd
+QEMU_BIOS_loongarch ?= edk2-loongarch64-code.fd
+# https://gitlab.com/qemu-project/qemu/-/blob/master/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin
+QEMU_BIOS_riscv32   ?= opensbi-riscv32-generic-fw_dynamic.bin
+QEMU_BIOS            = $(QEMU_BIOS_$(ARCH))
+QEMU_ARGS_BIOS       = $(if $(QEMU_BIOS),-bios $(QEMU_BIOS))
+
 # QEMU_ARGS : some arch-specific args to pass to qemu
 QEMU_ARGS_i386       = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_x86_64     = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
@@ -122,7 +130,7 @@ QEMU_ARGS_riscv64    = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_T
 QEMU_ARGS_riscv      = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_s390       = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_loongarch  = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
-QEMU_ARGS            = $(QEMU_ARGS_$(ARCH)) $(QEMU_ARGS_EXTRA)
+QEMU_ARGS            = $(QEMU_ARGS_$(ARCH)) $(QEMU_ARGS_BIOS) $(QEMU_ARGS_EXTRA)
 
 # OUTPUT is only set when run from the main makefile, otherwise
 # it defaults to this nolibc directory.
@@ -167,9 +175,15 @@ endif
 # allow run tests on all architectures: run-user-all, run-all (=run-default-all), run-tiny-all
 ARCHS   ?= $(shell sed -ne 's/^DEFCONFIG_\([^ ]*\) .*/\1/p' $(CURDIR)/Makefile)
 GOALS   ?= run-user run-tiny run-default
+export $(foreach a, $(ARCHS), QEMU_BIOS_$a)
 RUN_ALL ?= _t=$@; t=$${_t%-all}; [ "$$t" = "run" ] && t=run-default; \
 	   if echo $(GOALS) | grep -wq "$$t"; then \
-		for a in $(ARCHS); do echo "Testing $$t for $${a}:"; make $$t ARCH=$$a; cp $(CURDIR)/run.out $(CURDIR)/run-$$a.out; done; \
+		for a in $(ARCHS); do \
+			echo "Testing $$t for $${a}:"; \
+			eval bios=\$${QEMU_BIOS_$$a}; \
+			if [ -n "$${bios}" -a ! -f "$${bios}" ]; then echo "\nIgnoring $$a test, no bios: $${bios} found."; exit 0; fi; \
+			make $$t ARCH=$$a; cp $(CURDIR)/run.out $(CURDIR)/run-$$a.out; \
+		done; \
 		echo "\n\nTesting summary of $$t:\n"; \
 		for a in $(ARCHS); do echo $${a}:; echo; $(REPORT) $(CURDIR)/run-$$a.out; echo; done; \
 	   else \
Signature · 1 line
2.25.1
#23 lore raw

cover selftests/nolibc: add minimal kernel config support

In reply to Zhangjin Wu

Hi, Willy and Thomas

I'm planning to renew this patchset, so, require more discuss.

The main goal of this minimal kernel config support here is faster build and therefore faster test for the run target, but this old patchset is not enough, especially for the 'mrproper' operation during every config:

    defconfig:
     	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
     
    +tinyconfig:
    +	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare
    +

In order to avoid this 'mrproper', one solution is create an arch specicifc output directory for all of the kernel targets to avoid they share the same build output (by default, it is the same as the source code directory and therefore pollute the source code tree), it is required for both tools/nolibc and selftests/nolibc:

    // tools/nolibc

    headers_standalone: headers
    	$(Q)$(MAKE) -C $(srctree) headers
    	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
    

    // selftests/nolibc

    defconfig:
    	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
    
    kernel: initramfs
    	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs
    
    # run the tests after building the kernel
    run: kernel
    	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
    	$(Q)$(REPORT) $(CURDIR)/run.out

The "O" variable could be used to pass an arch specific output for every arch:

# OUTPUT is only set when run from the main makefile, otherwise # it defaults to this nolibc directory.
OUTPUT ?= $(CURDIR)/

# Architecture specific directory
ARCH_OUTPUT ?= $(CURDIR)/build/$(ARCH)/

With "O":

    // tools/nolibc

    KERNEL_BUILD   ?= $(ARCH_OUTPUT)linux/
    MAKE_KERNEL     = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH)

    headers_standalone: headers
    	$(Q)$(MAKE_KERNEL) headers
    	$(Q)$(MAKE_KERNEL) headers_install INSTALL_HDR_PATH=$(ARCH_OUTPUT)sysroot
    

    // selftests/nolibc
    RUN_OUT         = $(ARCH_OUTPUT)run.out
    ...
    NOLIBC_INITRAMFS  = $(ARCH_OUTPUT)initramfs/
    ...
    KERNEL_BUILD   ?= $(ARCH_OUTPUT)linux/
    MAKE_KERNEL     = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
    KERNEL_IMAGE    = $(KERNEL_BUILD)$(IMAGE)
    KERNEL_CONFIG   = $(KERNEL_BUILD).config

    defconfig:
    	$(Q)$(MAKE_KERNEL) $(DEFCONFIG) prepare
    
    kernel: initramfs
    	$(Q)$(MAKE_KERNEL) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(NOLIBC_INITRAMFS)
    
    # run the tests after building the kernel
    run: kernel
    	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(KERNEL_IMAGE)" -serial stdio $(QEMU_ARGS) > "$(RUN_OUT)"
    	$(Q)$(REPORT) $(RUN_OUT)

And further, the output of the other targets should be put in the $(ARCH_OUTPUT) directory too, include the sysroot, nolibc-test, initramfs, run.out and even libc-test, the whole Makefile of selftests/nolibc will be touched.

I have prepared and verified such a solution locally, before send the patches, welcome your suggestions.

My local patchset now provides such features:

- Allow build everything for a target arch in the arch specific directory

- Allow clean everything of a target arch

- Allow trigger a config target automatically for the run target, use the new tinyconfig by default, be able to choose defconfig as before

- Allow configure additional config options for both defconfig and tinyconfig

- Allow configure the default CROSS_COMPILE by default, only the ARCH variable is required

- Allow download the missing cross toolchains automatically (from mirrrors.edge.kernel.org)

- Allow download the missing bios automatically

- Allow download and build musl automatically

- Allow run tests for all of the supported architectures automatically with one command with minimal preparation, may be only the installation of qemu-user and qemu-system

With the new patchset, it is able to rebuild and rerun everything in several minutes, it may make people very happy to develop nolibc itself and also make people to work on linux with nolibc, especially for developing and testing a new kernel feature ;-)

Without this reorg of the output directory, it is very hard to verify this tinyconfig support patchset itself, it always require 'mrproper' to clean the whole build while config and test another ARCH (because the source code tree is polluted). for one ARCH, it is not that time-costed, but for all, it waste too much time. That means, without the arch specific output support, our minimal config support is really not that helpful, it also breaks the possibility to share the already built binaries for rebuild after a reconfig.

If it is hard to make decision on this description, I'm ok to send the whole new patchset for more discussion.

The change is 'huge' and the review work may be 'huge' too ;-)

Thanks,
Zhangjin

Quoted 147 lines

Willy, Thomas

We just sent the 'selftests/nolibc: allow run with minimal kernel config' series [1], Here is the 'minimal' kernel config support, with both of them, it is possible to run nolibc-test for all architectures with oneline command and in less than ~30 minutes - 1 hour (not fullly measured yet):

    // run with tiny config + qemu-system
    // Note: rv32 and loongarch require to download the bios at first
    $ time make run-tiny-all QUIET_RUN=1

    // run with default config + qemu-system
    $ time make run-default-all QUIET_RUN=1

    // run with qemu-user
    $ time make run-user-all QUIET_RUN=1

Besides the 'tinyconfig' suggestion from Thomas, this patch also merge the generic part of my local powerpc porting (the extconfig to add additional console support).

This is applied after the test report patchset [2] and the rv32 compile patchset [3], because all of them touched the same Makefile.

Even without the 'selftests/nolibc: allow run with minimal kernel config' series [1], all of the tests can pass except the /proc/self/net related ones (We haven't enable CONFIG_NET in this patchset), the chmod_net one will be removed by Thomas from this patchset [4] for the wrong chmodable attribute issue of /proc/self/net, the link_cross one can be simply fixed up by using another /proc/self interface (like /proc/self/cmdline), which will be covered in our revision of the [1] series.

Beside the core 'minimal' config support, some generic patch are added together to avoid patch conflicts.

* selftests/nolibc: add test for -include /path/to/nolibc.h

Add a test switch to allow run nolibc-test with nolibc.h

* selftests/nolibc: print result to the screen too

Let the run targets print results by default, allow disable by QUIET_RUN=1

* selftests/nolibc: allow use x86_64 toolchain for i386

Allow use x86_64 toolchains for i386

* selftests/nolibc: add menuconfig target for manual config

a new 'menuconfig' target added for development and debugging

* selftests/nolibc: add tinyconfig target

a new 'tinyconfig' compare to 'defconfig', smaller and faster, but not enough for boot and print, require following 'extconfig' target

* selftests/nolibc: allow customize extra kernel config options

a new 'extconfig' allows to add extra config options for 'defconfig' and 'tinyconfig'

* selftests/nolibc: add common extra config options selftests/nolibc: add power reset control support selftests/nolibc: add procfs, shmem and tmpfs

Add common extra configs, the 3rd one (procfs, shmem and tmpfs) can be completely reverted after [1] series, but as discuss with Thomas, procfs may be still a hard requirement.

* selftests/nolibc: add extra configs for i386 selftests/nolibc: add extra configs for x86_64 selftests/nolibc: add extra configs for arm64 selftests/nolibc: add extra configs for arm selftests/nolibc: add extra configs for mips selftests/nolibc: add extra configs for riscv32 selftests/nolibc: add extra configs for riscv64 selftests/nolibc: add extra configs for s390x selftests/nolibc: add extra configs for loongarch

Add architecture specific extra configs to let kernel boot and nolibc-test print. The rv32 added here is only for test, it should not be merged before the missing 64bit syscalls are added (still wait for the merging of the __sysret and -ENOSYS patches).

* selftests/nolibc: config default CROSS_COMPILE
selftests/nolibc: add run-tiny and run-default

both run-tiny and run-default are added to do config and run together, this easier test a log.

* selftests/nolibc: allow run tests on all targets selftests/nolibc: detect bios existing to avoid hang

Further allow do run-user, run-tiny and run-default for all architectures at once, the -all suffix is added to do so.

Since some generic patches are still in review, before sending the left rv32 patches, I'm will send more generic patches later, the coming one is arch-xxx.h cleanup, and then, the 32bit powerpc porting support.

For the compile speedup, the next step may be add architecture specific 'O' support, which may allow us rerun across architectures without mrproper, for a single architecture development, this 'minimal' config should be enough ;-)

Thanks.

Best regards,
Zhangjin
---
[1]: https://lore.kernel.org/lkml/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://lore.kernel.org/linux-riscv/[email protected]/
[4]: https://lore.kernel.org/lkml/[email protected]/

Zhangjin Wu (22):
selftests/nolibc: add test for -include /path/to/nolibc.h selftests/nolibc: print result to the screen too selftests/nolibc: allow use x86_64 toolchain for i386 selftests/nolibc: add menuconfig target for manual config selftests/nolibc: add tinyconfig target selftests/nolibc: allow customize extra kernel config options selftests/nolibc: add common extra config options selftests/nolibc: add power reset control support selftests/nolibc: add procfs, shmem and tmpfs selftests/nolibc: add extra configs for i386 selftests/nolibc: add extra configs for x86_64 selftests/nolibc: add extra configs for arm64 selftests/nolibc: add extra configs for arm selftests/nolibc: add extra configs for mips selftests/nolibc: add extra configs for riscv32 selftests/nolibc: add extra configs for riscv64 selftests/nolibc: add extra configs for s390x selftests/nolibc: add extra configs for loongarch selftests/nolibc: config default CROSS_COMPILE selftests/nolibc: add run-tiny and run-default selftests/nolibc: allow run tests on all targets selftests/nolibc: detect bios existing to avoid hang

 tools/testing/selftests/nolibc/Makefile | 125 ++++++++++++++++++++++--
 1 file changed, 119 insertions(+), 6 deletions(-)
Signature · 1 line
2.25.1
maintainer
#24 lore raw

Hi Zhangjin,

Quoted 8 lines · Zhangjin Wu

Hi, Willy and Thomas

I'm planning to renew this patchset, so, require more discuss.

The main goal of this minimal kernel config support here is faster build and therefore faster test for the run target, but this old patchset is not enough, especially for the 'mrproper' operation during every config:

Quoted 6 lines
    defconfig:
     	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
     
    +tinyconfig:
    +	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare
    +

It will depend on the number of runs, because the mrproper will only be run before recreating a config. Logically once you have your kernel you're not going to reconfigure it.

Quoted 5 lines

In order to avoid this 'mrproper', one solution is create an arch specicifc output directory for all of the kernel targets to avoid they share the same build output (by default, it is the same as the source code directory and therefore pollute the source code tree), it is required for both tools/nolibc and selftests/nolibc:

This could provide an improvement when building for multiple archs in a row, indeed. But keep in mind that an important aspect is also to be able to test the kernel you're currently working on, and not to have to rebuild it again for nolibc into a subdir.

And given that the makefile doesn't know how to iterate over multiple archs, it would make sense to permit to pass the build output as an argument to the makefile so that the external script calling make for each arch would just pass the build directory (as is sometimes done for the normal kernel).

Quoted 23 lines

The "O" variable could be used to pass an arch specific output for every arch:

# OUTPUT is only set when run from the main makefile, otherwise # it defaults to this nolibc directory.
OUTPUT ?= $(CURDIR)/

# Architecture specific directory
ARCH_OUTPUT ?= $(CURDIR)/build/$(ARCH)/

With "O":

    // tools/nolibc

    KERNEL_BUILD   ?= $(ARCH_OUTPUT)linux/
    MAKE_KERNEL     = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH)

    headers_standalone: headers
    	$(Q)$(MAKE_KERNEL) headers
    	$(Q)$(MAKE_KERNEL) headers_install INSTALL_HDR_PATH=$(ARCH_OUTPUT)sysroot
Quoted 7 lines
    // selftests/nolibc
    RUN_OUT         = $(ARCH_OUTPUT)run.out
    ...
    NOLIBC_INITRAMFS  = $(ARCH_OUTPUT)initramfs/
    ...
    KERNEL_BUILD   ?= $(ARCH_OUTPUT)linux/
    MAKE_KERNEL     = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)

Yes but this one will break reuse of the already built kernel. Also for those trying all archs for a single test it will eat a lot of disk space (when enabling certain debugging symbols you can end up with extremely large build dirs, I already ran out of space a few times).

I think that instead we should just let the user pass this "O=" to the make command and let the lower ones inherit it. That makes it closer to what it usually done. I'm also fine with using another variable if there's a good reason for this, but given that it's already called "O" it's not something unexpected to the user. Thus we could imagine just running:

    for arch in "${ARCHS[@]}"; do
        make -j$(nproc) O=$PWD/kernel-$arch ARCH=$arch \
             CROSS_COMPILE=/path/to/cc/$arch/bin/$arch- \
             run
    done

Which is simple enough without requiring a user to have to figure what build option to hack on for just a regular build.

Quoted 4 lines

And further, the output of the other targets should be put in the $(ARCH_OUTPUT) directory too, include the sysroot, nolibc-test, initramfs, run.out and even libc-test, the whole Makefile of selftests/nolibc will be touched.

It's a matter of taste but there are pros and cons. I personally spend my time using grep -r on the sysroots, having gigabytes of .o there would quickly made me regret that approach. I would also argue that when you pass "O=" to the kernel build, it serves as a prefix and the directory hierarchy is preserved, so if you would want to put the sysroots etc under $O then it would be more logical that it appears in $O/tools/testing/selftests/nolibc/sysroot/$arch, which would then remain compatible with today when O corresponds to the kernel's CURDIR. Then maybe that would be the solution to making O per-arch for those who want to re-run over many kernels, while keeping it easy for those who want to test their own kernel or want to grep in sysroot.

Quoted 16 lines

I have prepared and verified such a solution locally, before send the patches, welcome your suggestions.

My local patchset now provides such features:

- Allow build everything for a target arch in the arch specific directory

- Allow clean everything of a target arch

- Allow trigger a config target automatically for the run target, use the new tinyconfig by default, be able to choose defconfig as before

- Allow configure additional config options for both defconfig and tinyconfig

- Allow configure the default CROSS_COMPILE by default, only the ARCH variable is required

Quoted 8 lines

- Allow download the missing cross toolchains automatically (from mirrrors.edge.kernel.org)

- Allow download the missing bios automatically

- Allow download and build musl automatically

- Allow run tests for all of the supported architectures automatically with one command with minimal preparation, may be only the installation of qemu-user and qemu-system

There can be good things there, but I'm seeing a lot of "by default" and "automatically" which worry me, because when you're debugging something, there's nothing more annoying than all the stuff happening in your back and fooling your analysis. On the opposite I do value the ability to proceed through steps that I don't need to revalidate, and these ones can be done automatically just by chaining a few commands from a script.

And no download should ever be done automatically. Any download must always be deliberately and consciously initiated by the user. How many times we've seen some scripts stall with no information, just to discover a SYN_SENT socket in netstat! You can have a "download-toolchain", "download-bios", "download-musl" etc if you want, but quite frankly, I'm seriously doubting it will be *that* useful to anyone working on the kernel since by definition they already have these toolchains, and sometimes different ones (e.g. linaro, bootlin etc) and in different versions. In fact if some expresses the need for having a simplified command to download a toolchain, then it wouldn't be specific to nolibc-test but would be used as a general tool for the kernel since it would be shared with all the rest of the code. You could also imagine downloading and building qemu, but it also takes a while and we're very far from the simple build script whose purpose was to validate that a few syscalls appeared to work fine.

Quoted 4 lines

With the new patchset, it is able to rebuild and rerun everything in several minutes, it may make people very happy to develop nolibc itself and also make people to work on linux with nolibc, especially for developing and testing a new kernel feature ;-)

I doubt it. Rebuilding everything is not what most developers need. What they need is to just fix a missing parenthesis that broke the build and rebuild the strict minimum, restarting from where they left. And they need to be able to figure what caused that "FAILED" output, disassemble the output, re-run it manually under qemu-user, rebuild the .c manually after copy-pasting the whole command line via "V=1", etc.

Keep in mind that the purpose of a selftest is not to optimize the case where it works fine, but to optimize the developer's work when it fails. This is something that a lot of people tend to get wrong. They're proud of a "make world" that does everything including delivering them pizzas to wait for the builds to complete, and they say "hey, impressed?". But when someone else reports "I'm getting this strange error I don't understand", they can hardly suggest more than "hmmm run make clean, try again, or use the same distro as me because it works for me".

What I've tried to focus on from the beginning with this tiny test suite precisely is debugging. That's why I wanted to support alternate libcs, to help compare, making it easy to use a different toolchain and other cflags, running in userland, building just the binary so that I can scp it to a remote native machine etc. All of these are important, and your approach above makes me feel like I would lose most of it, or still be able to do that but in a very complicated way (i.e. constantly have to reverse-engineer a complicated makefile to figure where I can inject an option).

However one thing I mentioned in the past is the usefulness of some batching scripts (I even shared one). And I think that helping the user prepare certain steps or iterate over architectures *is* useful. When you do it in two layers (the script replacing the user, the makefile doing the build job), it remains easy and convenient to use, and you can pick only what you need (e.g. "please build musl for me"). And if something goes wrong, it's simple for the user to takeover and complete that failed task by changing an arch name, a directory or anything, and have their tools ready. Let's just never make that automatic for the sake of everyone's health!

Thanks,
Willy

#25 lore raw

Hi, Willy

Quoted 15 lines

Hi Zhangjin,

Quoted 11 lines · Zhangjin Wu

Hi, Willy and Thomas

I'm planning to renew this patchset, so, require more discuss.

The main goal of this minimal kernel config support here is faster build and therefore faster test for the run target, but this old patchset is not enough, especially for the 'mrproper' operation during every config:

    defconfig:
     	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
Quoted 8 lines
Quoted 3 lines
    +tinyconfig:
    +	$(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare
    +

It will depend on the number of runs, because the mrproper will only be run before recreating a config. Logically once you have your kernel you're not going to reconfigure it.

Yes, for one arch it is once.

Quoted 48 lines
Quoted 5 lines

In order to avoid this 'mrproper', one solution is create an arch specicifc output directory for all of the kernel targets to avoid they share the same build output (by default, it is the same as the source code directory and therefore pollute the source code tree), it is required for both tools/nolibc and selftests/nolibc:

This could provide an improvement when building for multiple archs in a row, indeed. But keep in mind that an important aspect is also to be able to test the kernel you're currently working on, and not to have to rebuild it again for nolibc into a subdir.

And given that the makefile doesn't know how to iterate over multiple archs, it would make sense to permit to pass the build output as an argument to the makefile so that the external script calling make for each arch would just pass the build directory (as is sometimes done for the normal kernel).

Quoted 23 lines

The "O" variable could be used to pass an arch specific output for every arch:

# OUTPUT is only set when run from the main makefile, otherwise # it defaults to this nolibc directory.
OUTPUT ?= $(CURDIR)/

# Architecture specific directory
ARCH_OUTPUT ?= $(CURDIR)/build/$(ARCH)/

With "O":

    // tools/nolibc

    KERNEL_BUILD   ?= $(ARCH_OUTPUT)linux/
    MAKE_KERNEL     = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH)

    headers_standalone: headers
    	$(Q)$(MAKE_KERNEL) headers
    	$(Q)$(MAKE_KERNEL) headers_install INSTALL_HDR_PATH=$(ARCH_OUTPUT)sysroot
Quoted 7 lines
    // selftests/nolibc
    RUN_OUT         = $(ARCH_OUTPUT)run.out
    ...
    NOLIBC_INITRAMFS  = $(ARCH_OUTPUT)initramfs/
    ...
    KERNEL_BUILD   ?= $(ARCH_OUTPUT)linux/
    MAKE_KERNEL     = $(MAKE) -C "$(srctree)" O="$(KERNEL_BUILD)" ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE)
Quoted 5 lines

Yes but this one will break reuse of the already built kernel. Also for those trying all archs for a single test it will eat a lot of disk space (when enabling certain debugging symbols you can end up with extremely large build dirs, I already ran out of space a few times).

For defconfig, it is, but for tinyconfig, it is smaller ;-)

Of course, with debugging symbols, it is bigger.

Quoted 8 lines

I think that instead we should just let the user pass this "O=" to the make command and let the lower ones inherit it. That makes it closer to what it usually done. I'm also fine with using another variable if there's a good reason for this, but given that it's already called "O" it's not something unexpected to the user. Thus we could imagine just running:

Quoted 9 lines
    for arch in "${ARCHS[@]}"; do
        make -j$(nproc) O=$PWD/kernel-$arch ARCH=$arch \
             CROSS_COMPILE=/path/to/cc/$arch/bin/$arch- \
             run
    done

Which is simple enough without requiring a user to have to figure what build option to hack on for just a regular build.

Yeah, passing it from a script is a substitute.

Quoted 5 lines
Quoted 4 lines

And further, the output of the other targets should be put in the $(ARCH_OUTPUT) directory too, include the sysroot, nolibc-test, initramfs, run.out and even libc-test, the whole Makefile of selftests/nolibc will be touched.

Quoted 12 lines

It's a matter of taste but there are pros and cons. I personally spend my time using grep -r on the sysroots, having gigabytes of .o there would quickly made me regret that approach. I would also argue that when you pass "O=" to the kernel build, it serves as a prefix and the directory hierarchy is preserved, so if you would want to put the sysroots etc under $O then it would be more logical that it appears in $O/tools/testing/selftests/nolibc/sysroot/$arch, which would then remain compatible with today when O corresponds to the kernel's CURDIR. Then maybe that would be the solution to making O per-arch for those who want to re-run over many kernels, while keeping it easy for those who want to test their own kernel or want to grep in sysroot.

Agree, using a $(ARCH_OUTPUT) does changes directory hierarchy.

Quoted 25 lines
Quoted 16 lines

I have prepared and verified such a solution locally, before send the patches, welcome your suggestions.

My local patchset now provides such features:

- Allow build everything for a target arch in the arch specific directory

- Allow clean everything of a target arch

- Allow trigger a config target automatically for the run target, use the new tinyconfig by default, be able to choose defconfig as before

- Allow configure additional config options for both defconfig and tinyconfig

- Allow configure the default CROSS_COMPILE by default, only the ARCH variable is required

Quoted 8 lines

- Allow download the missing cross toolchains automatically (from mirrrors.edge.kernel.org)

- Allow download the missing bios automatically

- Allow download and build musl automatically

- Allow run tests for all of the supported architectures automatically with one command with minimal preparation, may be only the installation of qemu-user and qemu-system

Quoted 7 lines

There can be good things there, but I'm seeing a lot of "by default" and "automatically" which worry me, because when you're debugging something, there's nothing more annoying than all the stuff happening in your back and fooling your analysis. On the opposite I do value the ability to proceed through steps that I don't need to revalidate, and these ones can be done automatically just by chaining a few commands from a script.

Ok, let these work in a standalone script.

Quoted 7 lines

And no download should ever be done automatically. Any download must always be deliberately and consciously initiated by the user. How many times we've seen some scripts stall with no information, just to discover a SYN_SENT socket in netstat! You can have a "download-toolchain", "download-bios", "download-musl" etc if you want, but quite frankly, I'm seriously doubting it will be *that* useful to anyone working on the kernel since by definition they already have these toolchains, and

Quoted 9 lines

sometimes different ones (e.g. linaro, bootlin etc) and in different versions. In fact if some expresses the need for having a simplified command to download a toolchain, then it wouldn't be specific to nolibc-test but would be used as a general tool for the kernel since it would be shared with all the rest of the code. You could also imagine downloading and building qemu, but it also takes a while and we're very far from the simple build script whose purpose was to validate that a few syscalls appeared to work fine.

More work may in reality add extra unexpected work outside of our core target.

Quoted 5 lines
Quoted 4 lines

With the new patchset, it is able to rebuild and rerun everything in several minutes, it may make people very happy to develop nolibc itself and also make people to work on linux with nolibc, especially for developing and testing a new kernel feature ;-)

Quoted 7 lines

I doubt it. Rebuilding everything is not what most developers need. What they need is to just fix a missing parenthesis that broke the build and rebuild the strict minimum, restarting from where they left. And they need to be able to figure what caused that "FAILED" output, disassemble the output, re-run it manually under qemu-user, rebuild the .c manually after copy-pasting the whole command line via "V=1", etc.

It is mainly for a cross arch change, like the _start_c() patchset we just discuss. This may also happen for every release (not that helpful, for a new release, a mrproper or distclean may be required).

Quoted 9 lines

Keep in mind that the purpose of a selftest is not to optimize the case where it works fine, but to optimize the developer's work when it fails. This is something that a lot of people tend to get wrong. They're proud of a "make world" that does everything including delivering them pizzas to wait for the builds to complete, and they say "hey, impressed?". But when someone else reports "I'm getting this strange error I don't understand", they can hardly suggest more than "hmmm run make clean, try again, or use the same distro as me because it works for me".

Parts of them do want to meet the 'optimize the developer's work when it fails', for example, new developers may be hard to find a loongarch or riscv bios, or find a toolchain for the not frequently used architecture, to avoid introduce many bug reports about "strange errors" outside of our core functions, perhaps we'd better do these in a nolibc doc under Documentation/, tell people how to prepare the develop and test environment of nolibc and how to use nolibc there.

Quoted 10 lines

What I've tried to focus on from the beginning with this tiny test suite precisely is debugging. That's why I wanted to support alternate libcs, to help compare, making it easy to use a different toolchain and other cflags, running in userland, building just the binary so that I can scp it to a remote native machine etc. All of these are important, and your approach above makes me feel like I would lose most of it, or still be able to do that but in a very complicated way (i.e. constantly have to reverse-engineer a complicated makefile to figure where I can inject an option).

Let's move most of them to a nolibc doc.

Quoted 2 lines

However one thing I mentioned in the past is the usefulness of some batching scripts (I even shared one).

Yeah, I remember, Thanks. I have used something similar in my Linux Lab project.

Quoted 9 lines

And I think that helping the user
prepare certain steps or iterate over architectures *is* useful. When you do it in two layers (the script replacing the user, the makefile doing the build job), it remains easy and convenient to use, and you can pick only what you need (e.g. "please build musl for me"). And if something goes wrong, it's simple for the user to takeover and complete that failed task by changing an arch name, a directory or anything, and have their tools ready. Let's just never make that automatic for the sake of everyone's health!

Ok, the revision will align with the original Makefile and remove the automatic parts and no change about the OUTPUT.

Best regards,
Zhangjin

Quoted 3 lines

Thanks,
Willy

maintainer
#26 lore raw
Quoted 13 lines · Zhangjin Wu
Quoted 5 lines
Quoted 4 lines

With the new patchset, it is able to rebuild and rerun everything in several minutes, it may make people very happy to develop nolibc itself and also make people to work on linux with nolibc, especially for developing and testing a new kernel feature ;-)

Quoted 7 lines

I doubt it. Rebuilding everything is not what most developers need. What they need is to just fix a missing parenthesis that broke the build and rebuild the strict minimum, restarting from where they left. And they need to be able to figure what caused that "FAILED" output, disassemble the output, re-run it manually under qemu-user, rebuild the .c manually after copy-pasting the whole command line via "V=1", etc.

Quoted 3 lines

It is mainly for a cross arch change, like the _start_c() patchset we just discuss. This may also happen for every release (not that helpful, for a new release, a mrproper or distclean may be required).

Maybe, but let's focus on fixing what's really annoying before trying to improve things that work. If you find yourself failing to do certain things, annoyed with some flags that are forced on you etc, not finding a way to work with multiple output dirs, these are good reasons for applying changes. But thinking that it will likely be better organized this or that way tends to make us forget what we're relying on and that we might lose by accident.

Quoted 10 lines
Quoted 9 lines

Keep in mind that the purpose of a selftest is not to optimize the case where it works fine, but to optimize the developer's work when it fails. This is something that a lot of people tend to get wrong. They're proud of a "make world" that does everything including delivering them pizzas to wait for the builds to complete, and they say "hey, impressed?". But when someone else reports "I'm getting this strange error I don't understand", they can hardly suggest more than "hmmm run make clean, try again, or use the same distro as me because it works for me".

Quoted 7 lines

Parts of them do want to meet the 'optimize the developer's work when it fails', for example, new developers may be hard to find a loongarch or riscv bios, or find a toolchain for the not frequently used architecture, to avoid introduce many bug reports about "strange errors" outside of our core functions, perhaps we'd better do these in a nolibc doc under Documentation/, tell people how to prepare the develop and test environment of nolibc and how to use nolibc there.

Reading the beginning of the sentence made me immediately think that it's what doc is for. You know, if you give a fish to a hungry man he eats one day, if you teach him to fish he eats every day. Knowing what to download from where is much more instructive than running "make download" or even worse, "make" and having the downloads secretly succeed (or fail). If you think the doc is hard to find I'm also fine with a patch for makefile and/or nolibc-test passing a pointer to its location as a reminding comment for example.

Quoted 10 lines
Quoted 9 lines

And I think that helping the user
prepare certain steps or iterate over architectures *is* useful. When you do it in two layers (the script replacing the user, the makefile doing the build job), it remains easy and convenient to use, and you can pick only what you need (e.g. "please build musl for me"). And if something goes wrong, it's simple for the user to takeover and complete that failed task by changing an arch name, a directory or anything, and have their tools ready. Let's just never make that automatic for the sake of everyone's health!

Quoted 2 lines

Ok, the revision will align with the original Makefile and remove the automatic parts and no change about the OUTPUT.

Just check that you can force it from your script on the make command line. If you see that it's not possible, we should do something because I don't want to force you to make distclean all the time if not needed. But if you find that passing certain options (O=, OUTPUT= or anything else) does the job, it only needs to be documented.

Thanks,
Willy

#27 lore raw

Hi, Willy, Thomas

I have prepared the powerpc + powerpc64 support together with the tinyconfig for them, still have some questions for more discussion.

Quoted 1 line · Zhangjin Wu

[...]

Quoted 10 lines

Reading the beginning of the sentence made me immediately think that it's what doc is for. You know, if you give a fish to a hungry man he eats one day, if you teach him to fish he eats every day. Knowing what to download from where is much more instructive than running "make download" or even worse, "make" and having the downloads secretly succeed (or fail). If you think the doc is hard to find I'm also fine with a patch for makefile and/or nolibc-test passing a pointer to its location as a reminding comment for example.

The whole tinyconfig support for every architecture is 'huge', I plan to send them one by one, if required, will document them with the required bios and/or toolchain.

The first architectures plan to support are powerpc + powerpc64, powerpc does require extra kernel config options even with defconfig, so, it is a very good first example, and the extconfig target will be added together.

The left question from me is that if is it ok to just use tinyconfig instead of defconfig after we enable tinyconfig for a new architecture, I mean just add a new DEFCONFIG_<ARCH>=tinyconfig line for the new architecture, don't use the 'defconfig' any more, let's take a look at the powerpc/powerpc64 lines below:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = defconfig
    DEFCONFIG_x86_64     = defconfig
    DEFCONFIG_x86        = defconfig
    DEFCONFIG_arm64      = defconfig
    DEFCONFIG_arm        = multi_v7_defconfig
    DEFCONFIG_mips       = malta_defconfig
    DEFCONFIG_powerpc    = tinyconfig
    DEFCONFIG_powerpc64  = tinyconfig
    DEFCONFIG_riscv      = defconfig
    DEFCONFIG_s390       = defconfig
    DEFCONFIG_loongarch  = defconfig
    DEFCONFIG            = $(DEFCONFIG_$(XARCH))

Of course, we need to customize the EXTCONFIG for them (about ~5 options for all of the architectures):

    # extra kernel configs by architecture
    EXTCONFIG_powerpc    = $(addprefix -e ,COMPAT_32BIT_TIME PPC_PMAC PPC_OF_BOOT_TRAMPOLINE SERIAL_PMACZILOG SERIAL_PMACZILOG_TTYS SERIAL_PMACZILOG_CONSOLE)
    EXTCONFIG_powerpc64  = $(addprefix -e ,PPC64 CPU_LITTLE_ENDIAN PPC_POWERNV HVC_OPAL)
    EXTCONFIG_XARCH      = $(EXTCONFIG_$(XARCH))

The extra common options (based on default kernel tinyconfig) are also required to make nolibc-test.c passes without failures (~2 skips are procfs related, so, procfs is not added) are minimal:

    # extra kernel configs shared among architectures
    EXTCONFIG_COMMON     = -e BLK_DEV_INITRD --set-str INITRAMFS_SOURCE $(CURDIR)/initramfs
    EXTCONFIG_COMMON    += -e BINFMT_ELF
    EXTCONFIG_COMMON    += -e PRINTK -e TTY

Compare to defconfig, tinyconfig not only allows test all of the nolibc functions, but also is faster (around ~1-2 minutes, defconfig may cost ~30 minutes and even more) and brings us with smaller image size.

To only test nolibc itself, I do think tinyconfig with the above extconfig support is enough, even if we need more, we can update the EXTCONFIG_COMMON and EXTCONFIG_<ARCH> in the future.

I have prepared tinyconfig for all of the supported architectures locally, If you agree with only reserve the DEFCONFIG_<ARCH>=tinyconfig line, I will send a series of patchset to add tinyconfig for every architecture with it, at last, it will become:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = tinyconfig
    DEFCONFIG_x86_64     = tinyconfig
    DEFCONFIG_x86        = tinyconfig
    DEFCONFIG_arm64      = tinyconfig
    DEFCONFIG_arm        = tinyconfig
    DEFCONFIG_mips       = tinyconfig
    DEFCONFIG_powerpc    = tinyconfig
    DEFCONFIG_powerpc64  = tinyconfig
    DEFCONFIG_riscv      = tinyconfig
    DEFCONFIG_s390       = tinyconfig
    DEFCONFIG_loongarch  = tinyconfig
    DEFCONFIG            = $(DEFCONFIG_$(XARCH))

So, perhaps it is better to simply use tinyconfig as the default DEFCONFIG, and therefore there is no need to add powerpc and powerpc64 specific lines:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = defconfig
    DEFCONFIG_x86_64     = defconfig
    DEFCONFIG_x86        = defconfig
    DEFCONFIG_arm64      = defconfig
    DEFCONFIG_arm        = multi_v7_defconfig
    DEFCONFIG_mips       = malta_defconfig
    DEFCONFIG_riscv      = defconfig
    DEFCONFIG_s390       = defconfig
    DEFCONFIG_loongarch  = defconfig
    DEFCONFIG            = $(or $(DEFCONFIG_$(XARCH)),tinyconfig)

To support tinyconfig for a new architecture, we can simply remove the 'DEFCONFIG_<ARCH> = defconfig' line and get the core options from defconfig to customize the EXTCONFIG_ARCH, with tinyconfig, it is very fast and easy to verify the run target for a new architecture.

At last, we will have many EXTCONFIG_<ARCH> lines and only a DEFCONFIG line:

    # default kernel configurations that appear to be usable
    DEFCONFIG            = $(or $(DEFCONFIG_$(XARCH)),tinyconfig)

Or at last, we remove the above line and the defconfig target and only reserve a tinyconfig target:

    tinyconfig:
	$(Q)$(MAKE_KERNEL) tinyconfig prepare

Welcome your suggestion.

Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

And I think that helping the user
prepare certain steps or iterate over architectures *is* useful. When you do it in two layers (the script replacing the user, the makefile doing the build job), it remains easy and convenient to use, and you can pick only what you need (e.g. "please build musl for me"). And if something goes wrong, it's simple for the user to takeover and complete that failed task by changing an arch name, a directory or anything, and have their tools ready. Let's just never make that automatic for the sake of everyone's health!

Quoted 8 lines
Quoted 2 lines

Ok, the revision will align with the original Makefile and remove the automatic parts and no change about the OUTPUT.

Just check that you can force it from your script on the make command line. If you see that it's not possible, we should do something because I don't want to force you to make distclean all the time if not needed. But if you find that passing certain options (O=, OUTPUT= or anything else) does the job, it only needs to be documented.

Yeah, I have used objtree intead of srctree to fix up the O= argument support, it fills my requirement to build kernel for every architecture in their own output directory.

Best regards,
Zhangjin

Quoted 3 lines

Thanks,
Willy

#28 lore raw
Quoted 18 lines · Zhangjin Wu

Hi, Willy, Thomas

I have prepared the powerpc + powerpc64 support together with the tinyconfig for them, still have some questions for more discussion.

Quoted 1 line · Zhangjin Wu

[...]

Quoted 10 lines

Reading the beginning of the sentence made me immediately think that it's what doc is for. You know, if you give a fish to a hungry man he eats one day, if you teach him to fish he eats every day. Knowing what to download from where is much more instructive than running "make download" or even worse, "make" and having the downloads secretly succeed (or fail). If you think the doc is hard to find I'm also fine with a patch for makefile and/or nolibc-test passing a pointer to its location as a reminding comment for example.

Quoted 3 lines

The whole tinyconfig support for every architecture is 'huge', I plan to send them one by one, if required, will document them with the required bios and/or toolchain.

Which part is huge?
This is surprising.

Quoted 3 lines

The first architectures plan to support are powerpc + powerpc64, powerpc does require extra kernel config options even with defconfig, so, it is a very good first example, and the extconfig target will be added together.

Are you planning to do powerpc and tinyconfig support in one series? Splitting it would be better in my opinion.

Quoted 19 lines

The left question from me is that if is it ok to just use tinyconfig instead of defconfig after we enable tinyconfig for a new architecture, I mean just add a new DEFCONFIG_<ARCH>=tinyconfig line for the new architecture, don't use the 'defconfig' any more, let's take a look at the powerpc/powerpc64 lines below:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = defconfig
    DEFCONFIG_x86_64     = defconfig
    DEFCONFIG_x86        = defconfig
    DEFCONFIG_arm64      = defconfig
    DEFCONFIG_arm        = multi_v7_defconfig
    DEFCONFIG_mips       = malta_defconfig
    DEFCONFIG_powerpc    = tinyconfig
    DEFCONFIG_powerpc64  = tinyconfig
    DEFCONFIG_riscv      = defconfig
    DEFCONFIG_s390       = defconfig
    DEFCONFIG_loongarch  = defconfig
    DEFCONFIG            = $(DEFCONFIG_$(XARCH))
Quoted 7 lines

Of course, we need to customize the EXTCONFIG for them (about ~5 options for all of the architectures):

    # extra kernel configs by architecture
    EXTCONFIG_powerpc    = $(addprefix -e ,COMPAT_32BIT_TIME PPC_PMAC PPC_OF_BOOT_TRAMPOLINE SERIAL_PMACZILOG SERIAL_PMACZILOG_TTYS SERIAL_PMACZILOG_CONSOLE)
    EXTCONFIG_powerpc64  = $(addprefix -e ,PPC64 CPU_LITTLE_ENDIAN PPC_POWERNV HVC_OPAL)
    EXTCONFIG_XARCH      = $(EXTCONFIG_$(XARCH))

These could also be put into dedicated config files. Then they don't clutter the makefile and it's easier to maintain.

nolibc.config:       Generic configs on top of tinyconfig
nolibc.arm64.config: Arch-specific configs for arm64

Also the extra parameter could also be passed via command line arguments to make. This way we don't have to modify the configuration options at all. The user can provide a config (or we use a tinyconfig) and everything required by nolibc-test is enabled on top when building.

make CONFIG_COMPAT_32BIT_TIME=y ...

tools/testing/selftests/wireguard/qemu/ seems to be doing something similar as nolibc-test.

Quoted 9 lines

The extra common options (based on default kernel tinyconfig) are also required to make nolibc-test.c passes without failures (~2 skips are procfs related, so, procfs is not added) are minimal:

    # extra kernel configs shared among architectures
    EXTCONFIG_COMMON     = -e BLK_DEV_INITRD --set-str INITRAMFS_SOURCE $(CURDIR)/initramfs
    EXTCONFIG_COMMON    += -e BINFMT_ELF
    EXTCONFIG_COMMON    += -e PRINTK -e TTY
Quoted 7 lines

Compare to defconfig, tinyconfig not only allows test all of the nolibc functions, but also is faster (around ~1-2 minutes, defconfig may cost ~30 minutes and even more) and brings us with smaller image size.

To only test nolibc itself, I do think tinyconfig with the above extconfig support is enough, even if we need more, we can update the EXTCONFIG_COMMON and EXTCONFIG_<ARCH> in the future.

IMO tinyconfig is enough, defconfig doesn't seem to be necessary then.

Quoted 44 lines

I have prepared tinyconfig for all of the supported architectures locally, If you agree with only reserve the DEFCONFIG_<ARCH>=tinyconfig line, I will send a series of patchset to add tinyconfig for every architecture with it, at last, it will become:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = tinyconfig
    DEFCONFIG_x86_64     = tinyconfig
    DEFCONFIG_x86        = tinyconfig
    DEFCONFIG_arm64      = tinyconfig
    DEFCONFIG_arm        = tinyconfig
    DEFCONFIG_mips       = tinyconfig
    DEFCONFIG_powerpc    = tinyconfig
    DEFCONFIG_powerpc64  = tinyconfig
    DEFCONFIG_riscv      = tinyconfig
    DEFCONFIG_s390       = tinyconfig
    DEFCONFIG_loongarch  = tinyconfig
    DEFCONFIG            = $(DEFCONFIG_$(XARCH))

So, perhaps it is better to simply use tinyconfig as the default DEFCONFIG, and therefore there is no need to add powerpc and powerpc64 specific lines:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = defconfig
    DEFCONFIG_x86_64     = defconfig
    DEFCONFIG_x86        = defconfig
    DEFCONFIG_arm64      = defconfig
    DEFCONFIG_arm        = multi_v7_defconfig
    DEFCONFIG_mips       = malta_defconfig
    DEFCONFIG_riscv      = defconfig
    DEFCONFIG_s390       = defconfig
    DEFCONFIG_loongarch  = defconfig
    DEFCONFIG            = $(or $(DEFCONFIG_$(XARCH)),tinyconfig)

To support tinyconfig for a new architecture, we can simply remove the 'DEFCONFIG_<ARCH> = defconfig' line and get the core options from defconfig to customize the EXTCONFIG_ARCH, with tinyconfig, it is very fast and easy to verify the run target for a new architecture.

At last, we will have many EXTCONFIG_<ARCH> lines and only a DEFCONFIG line:

    # default kernel configurations that appear to be usable
    DEFCONFIG            = $(or $(DEFCONFIG_$(XARCH)),tinyconfig)
Quoted 7 lines

Or at last, we remove the above line and the defconfig target and only reserve a tinyconfig target:

    tinyconfig:
	$(Q)$(MAKE_KERNEL) tinyconfig prepare

Welcome your suggestion.

Looks fine to me either way.

Quoted 29 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

And I think that helping the user
prepare certain steps or iterate over architectures *is* useful. When you do it in two layers (the script replacing the user, the makefile doing the build job), it remains easy and convenient to use, and you can pick only what you need (e.g. "please build musl for me"). And if something goes wrong, it's simple for the user to takeover and complete that failed task by changing an arch name, a directory or anything, and have their tools ready. Let's just never make that automatic for the sake of everyone's health!

Quoted 8 lines
Quoted 2 lines

Ok, the revision will align with the original Makefile and remove the automatic parts and no change about the OUTPUT.

Just check that you can force it from your script on the make command line. If you see that it's not possible, we should do something because I don't want to force you to make distclean all the time if not needed. But if you find that passing certain options (O=, OUTPUT= or anything else) does the job, it only needs to be documented.

Yeah, I have used objtree intead of srctree to fix up the O= argument support, it fills my requirement to build kernel for every architecture in their own output directory.

Best regards,
Zhangjin

Quoted 3 lines

Thanks,
Willy

maintainer
#29 lore raw

Hi Zhangjin, Thomas,

Just a quick response below on one point.

Quoted 6 lines · Thomas Weißschuh
Quoted 3 lines

The first architectures plan to support are powerpc + powerpc64, powerpc does require extra kernel config options even with defconfig, so, it is a very good first example, and the extconfig target will be added together.

Are you planning to do powerpc and tinyconfig support in one series? Splitting it would be better in my opinion.

I agree for splitting.

(...)

Quoted 5 lines
Quoted 3 lines

To only test nolibc itself, I do think tinyconfig with the above extconfig support is enough, even if we need more, we can update the EXTCONFIG_COMMON and EXTCONFIG_<ARCH> in the future.

IMO tinyconfig is enough, defconfig doesn't seem to be necessary then.

This is a point where I think I disagree. In fact the important feature of defconfig, for any arch, is that it's supposed to work out of the box. If we need to add extra options on top of tiny config, someone will regularly have to maintain this set of options up to date when more are added or when they're split. It could even start to fire back to a few developers who want to make the config more modular and are forced to update these ones without knowing well how to proceed with nolibc. Thus I would suggest that defconfig remains available (as a make target) for when we want to rely on a safe config, and that tinyconfig + local tuning is available for those like us who tend to spend more time on nolibc and who don't care much about having to tweak some options once in a while.

Just my two cents,
Willy

#30 lore raw
In reply to Thomas Weißschuh

Hi, Thomas

Quoted 19 lines · Zhangjin Wu
Quoted 18 lines

Hi, Willy, Thomas

I have prepared the powerpc + powerpc64 support together with the tinyconfig for them, still have some questions for more discussion.

Quoted 1 line · Zhangjin Wu

[...]

Quoted 10 lines

Reading the beginning of the sentence made me immediately think that it's what doc is for. You know, if you give a fish to a hungry man he eats one day, if you teach him to fish he eats every day. Knowing what to download from where is much more instructive than running "make download" or even worse, "make" and having the downloads secretly succeed (or fail). If you think the doc is hard to find I'm also fine with a patch for makefile and/or nolibc-test passing a pointer to its location as a reminding comment for example.

Quoted 5 lines
Quoted 3 lines

The whole tinyconfig support for every architecture is 'huge', I plan to send them one by one, if required, will document them with the required bios and/or toolchain.

Which part is huge?

Actually, we only need to add ~30 lines for all of them ;-)

But some architectures require extra operations, for example,

- x86 requires extra "tsc=unstable noapic" kernel cmdline to save the CONFIG_ACPI options, if we only reserve tinyconfig target, it is ok to add it directly to the original QEMU_ARGS_<ARCH>

- x86 and some other architectures may fail to poweroff due to save some time-cost options or due to missing kernel poweroff support, they require qemu-poweroff timeout support

- some architectures have no built-in firmware for qemu, we need to document them carefully

- some architectures have no available toolchain in any Linux distributions, we need to document them carefully

...

Quoted 2 lines

This is surprising.

For one architecture, it is not that hard, but for all of the supported architectures, weird bugs came out one by one ;-)

Quoted 7 lines
Quoted 3 lines

The first architectures plan to support are powerpc + powerpc64, powerpc does require extra kernel config options even with defconfig, so, it is a very good first example, and the extconfig target will be added together.

Are you planning to do powerpc and tinyconfig support in one series? Splitting it would be better in my opinion.

Ok, as Willy replied in another thread, let's add powerpc support with defconfig as the first patchset.

Since, even with defconfig, we still need extra console options to make the test print normally, so, the extra options support code will be added anyway, adding tinyconfig support is a fastforward step.

Quoted 27 lines
Quoted 19 lines

The left question from me is that if is it ok to just use tinyconfig instead of defconfig after we enable tinyconfig for a new architecture, I mean just add a new DEFCONFIG_<ARCH>=tinyconfig line for the new architecture, don't use the 'defconfig' any more, let's take a look at the powerpc/powerpc64 lines below:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = defconfig
    DEFCONFIG_x86_64     = defconfig
    DEFCONFIG_x86        = defconfig
    DEFCONFIG_arm64      = defconfig
    DEFCONFIG_arm        = multi_v7_defconfig
    DEFCONFIG_mips       = malta_defconfig
    DEFCONFIG_powerpc    = tinyconfig
    DEFCONFIG_powerpc64  = tinyconfig
    DEFCONFIG_riscv      = defconfig
    DEFCONFIG_s390       = defconfig
    DEFCONFIG_loongarch  = defconfig
    DEFCONFIG            = $(DEFCONFIG_$(XARCH))
Quoted 7 lines

Of course, we need to customize the EXTCONFIG for them (about ~5 options for all of the architectures):

    # extra kernel configs by architecture
    EXTCONFIG_powerpc    = $(addprefix -e ,COMPAT_32BIT_TIME PPC_PMAC PPC_OF_BOOT_TRAMPOLINE SERIAL_PMACZILOG SERIAL_PMACZILOG_TTYS SERIAL_PMACZILOG_CONSOLE)
    EXTCONFIG_powerpc64  = $(addprefix -e ,PPC64 CPU_LITTLE_ENDIAN PPC_POWERNV HVC_OPAL)
    EXTCONFIG_XARCH      = $(EXTCONFIG_$(XARCH))
Quoted 6 lines

These could also be put into dedicated config files. Then they don't clutter the makefile and it's easier to maintain.

nolibc.config:       Generic configs on top of tinyconfig
nolibc.arm64.config: Arch-specific configs for arm64

Thanks, xxx.config files should work well with the merge tool:
scripts/kconfig/merge_config.sh.

Since in all of the supported architectures, the maximum extra options required by tinyconfig is 8, the others is around 5 options (some even less), all of them can be put in one line, at the same time, the initramfs source can use the same EXTCONFIG_COMMON logic, so, the oneline EXTCONFIG_<ARCH> is used without any new files.

Will take a look at the xxx.config files method.

But unfortunately, powerpc defconfig doesn't print by default, it also require extra console options, to apply the xxx.config files method, we need wait more time to restruct the powerpc support.

Quoted 7 lines

Also the extra parameter could also be passed via command line arguments to make. This way we don't have to modify the configuration options at all. The user can provide a config (or we use a tinyconfig) and everything required by nolibc-test is enabled on top when building.

make CONFIG_COMPAT_32BIT_TIME=y ...

This doesn't work sometimes especially when we want to disable an option, that's why we eventually use scripts/config instead.

Quoted 16 lines

tools/testing/selftests/wireguard/qemu/ seems to be doing something similar as nolibc-test.

Quoted 13 lines

The extra common options (based on default kernel tinyconfig) are also required to make nolibc-test.c passes without failures (~2 skips are procfs related, so, procfs is not added) are minimal:

    # extra kernel configs shared among architectures
    EXTCONFIG_COMMON     = -e BLK_DEV_INITRD --set-str INITRAMFS_SOURCE $(CURDIR)/initramfs
    EXTCONFIG_COMMON    += -e BINFMT_ELF
    EXTCONFIG_COMMON    += -e PRINTK -e TTY

Compare to defconfig, tinyconfig not only allows test all of the nolibc functions, but also is faster (around ~1-2 minutes, defconfig may cost ~30 minutes and even more) and brings us with smaller image size.

Quoted 6 lines
Quoted 3 lines

To only test nolibc itself, I do think tinyconfig with the above extconfig support is enough, even if we need more, we can update the EXTCONFIG_COMMON and EXTCONFIG_<ARCH> in the future.

IMO tinyconfig is enough, defconfig doesn't seem to be necessary then.

As Willy replied in another thread, tinyconfig may fail sometimes because many developers only use defconfig for tests, I did find tinyconfig fail on powerpc [1] and riscv64 [2], although later, both of them have been fixed.

I reported the 2nd one, the 1st one has just been fixed up by somebody else before I plan to send the patch, so, our nolibc tinyconfig support may function as a very good coverage test for kernel ;-)

[1]: https://lore.kernel.org/lkml/20230629124500.1.I55e2f4e7903d686c4484cb23c033c6a9e1a9d4c4@changeid/
[2]: https://lore.kernel.org/linux-riscv/[email protected]/

Quoted 47 lines
Quoted 47 lines

I have prepared tinyconfig for all of the supported architectures locally, If you agree with only reserve the DEFCONFIG_<ARCH>=tinyconfig line, I will send a series of patchset to add tinyconfig for every architecture with it, at last, it will become:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = tinyconfig
    DEFCONFIG_x86_64     = tinyconfig
    DEFCONFIG_x86        = tinyconfig
    DEFCONFIG_arm64      = tinyconfig
    DEFCONFIG_arm        = tinyconfig
    DEFCONFIG_mips       = tinyconfig
    DEFCONFIG_powerpc    = tinyconfig
    DEFCONFIG_powerpc64  = tinyconfig
    DEFCONFIG_riscv      = tinyconfig
    DEFCONFIG_s390       = tinyconfig
    DEFCONFIG_loongarch  = tinyconfig
    DEFCONFIG            = $(DEFCONFIG_$(XARCH))

So, perhaps it is better to simply use tinyconfig as the default DEFCONFIG, and therefore there is no need to add powerpc and powerpc64 specific lines:

    # default kernel configurations that appear to be usable
    DEFCONFIG_i386       = defconfig
    DEFCONFIG_x86_64     = defconfig
    DEFCONFIG_x86        = defconfig
    DEFCONFIG_arm64      = defconfig
    DEFCONFIG_arm        = multi_v7_defconfig
    DEFCONFIG_mips       = malta_defconfig
    DEFCONFIG_riscv      = defconfig
    DEFCONFIG_s390       = defconfig
    DEFCONFIG_loongarch  = defconfig
    DEFCONFIG            = $(or $(DEFCONFIG_$(XARCH)),tinyconfig)

To support tinyconfig for a new architecture, we can simply remove the 'DEFCONFIG_<ARCH> = defconfig' line and get the core options from defconfig to customize the EXTCONFIG_ARCH, with tinyconfig, it is very fast and easy to verify the run target for a new architecture.

At last, we will have many EXTCONFIG_<ARCH> lines and only a DEFCONFIG line:

    # default kernel configurations that appear to be usable
    DEFCONFIG            = $(or $(DEFCONFIG_$(XARCH)),tinyconfig)

Or at last, we remove the above line and the defconfig target and only reserve a tinyconfig target:

Quoted 6 lines
Quoted 4 lines
    tinyconfig:
	$(Q)$(MAKE_KERNEL) tinyconfig prepare

Welcome your suggestion.

Looks fine to me either way.

Ok, so, let's reserve defconfig as a backup although tinyconfig should work at most of the time.

Best regards,
Zhangjin