Skip to contentSkip to conversation

tools/nolibc: install unified multi-arch headers

RFC PATCH Superseded Willy Tarreau 3 participants 20 messages Jun 20, 2025 – Jun 27, 2025 Nolibc header file

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Willy Tarreau

author maintainer
#0 lore raw

cover tools/nolibc: install unified multi-arch headers

While the in-tree nolibc directory supports arch auto-detection, installing it is only performed for a single architecture at once, requiring as many sysroots as desired architectures. It is not convenient because external tools rarely expect to change their include path based on the target.

Furthermore, when installing it along with the UAPI headers for use with the nolibc toolchains, the issue is reinforced as UAPI may only be installed for a single architecture, and in this case we cannot even count on the headers that would have been packaged along with the compiler and its accompanying libc.

This patchset proposes a convenient approach to satisfy all this.

First, it adds a "headers_all_archs" target to nolibc, that will install the headers for all supported archs, so that they appear just like in the original source tree, meaning that when uapi headers are available (e.g. provided with the toolchain), the build will succeed for all suppored archs from a single installation directory.

Second, a new "install_all_archs" target iterates over all supported archs to install the corresponding UAPI headers, and moves each arch-specific "asm/" subdir to "asm-arch-$arch", then automatically recreates the files under asm/ to include the corresponding files based on the detected architecture. This results in a unified sysroot that can be used by any of the supported architectures without changing the include path.

I'm marking this as RFC because the operations are entirely performed inside the nolibc Makefile, and I'm wondering whether there could be any interest in generalizing the principle and moving it to the generic uapi installation itself. However I don't see how this could be easily done in this case, because here we have no other option but iterate over all supported architectures, and iterating over multiple archs is not something standard in the kbuild system. But we could also imagine having a script under scripts/ to install UAPI headers for all archs at once for example, so ideas are welcome. Of course if there's no perceived interest in generalizing this to uapi then it can stay in nolibc where the maintenance cost should remain quite low anyway.

Thanks for any comments!
Willy

--
Willy Tarreau (3):
tools/nolibc: merge i386 and x86_64 into a single x86 arch tools/nolibc: add a new target "headers_all_archs" to loop over all
archs
tools/nolibc: add a new "install_all_archs" target

 tools/include/nolibc/Makefile                 |  59 +++++-
 tools/include/nolibc/arch-i386.h              | 178 -----------------
 .../nolibc/{arch-x86_64.h => arch-x86.h}      | 180 +++++++++++++++++-
 tools/include/nolibc/arch.h                   |   6 +-
 4 files changed, 228 insertions(+), 195 deletions(-)
 delete mode 100644 tools/include/nolibc/arch-i386.h
 rename tools/include/nolibc/{arch-x86_64.h => arch-x86.h} (53%)
Signature · 1 line
2.17.5
author maintainer
#1 lore raw

RFC PATCH 1/3 tools/nolibc: merge i386 and x86_64 into a single x86 arch

This remained the only exception to the kernel's architectures organization and it's always a bit cumbersome to deal with. Let's merge i386 and x86_64 into x86. This will result in a single arch-x86.h file by default, and we'll no longer need to merge the two manually during installation. Requesting either i386 or x86_64 will also result in installing x86.

---
 tools/include/nolibc/Makefile                 |  10 +-
 tools/include/nolibc/arch-i386.h              | 178 -----------------
 .../nolibc/{arch-x86_64.h => arch-x86.h}      | 180 +++++++++++++++++-
 tools/include/nolibc/arch.h                   |   6 +-
 4 files changed, 179 insertions(+), 195 deletions(-)
 delete mode 100644 tools/include/nolibc/arch-i386.h
 rename tools/include/nolibc/{arch-x86_64.h => arch-x86.h} (53%)
tools/include/nolibc/Makefile +2 -8
index 823bfa08c6756..9197c79b267a4 100644
@@ -93,18 +93,12 @@ help:
 	@echo "  OUTPUT  = $(OUTPUT)"
 	@echo ""
 
-# Note: when ARCH is "x86" we concatenate both x86_64 and i386
 headers:
 	$(Q)mkdir -p $(OUTPUT)sysroot
 	$(Q)mkdir -p $(OUTPUT)sysroot/include
 	$(Q)cp --parents $(all_files) $(OUTPUT)sysroot/include/
-	$(Q)if [ "$(ARCH)" = "x86" ]; then      \
-		sed -e                          \
-		  's,^#ifndef _NOLIBC_ARCH_X86_64_H,#if !defined(_NOLIBC_ARCH_X86_64_H) \&\& defined(__x86_64__),' \
-		  arch-x86_64.h;                \
-		sed -e                          \
-		  's,^#ifndef _NOLIBC_ARCH_I386_H,#if !defined(_NOLIBC_ARCH_I386_H) \&\& !defined(__x86_64__),' \
-		  arch-i386.h;                  \
+	$(Q)if [ "$(ARCH)" = "i386" -o "$(ARCH)" = "x86_64" ]; then \
+		cat arch-x86.h;                 \
 	elif [ -e "$(arch_file)" ]; then        \
 		cat $(arch_file);               \
 	else                                    \
tools/include/nolibc/arch-i386.h deleted +0 -178

178 lines not on this page · raw message

tools/include/nolibc/arch-x86.h renamed from tools/include/nolibc/arch-x86_64.h +175 -5

194 lines not on this page · raw message

tools/include/nolibc/arch.h +2 -4
index d20b2304aac21..4ae57aaf97796 100644
@@ -15,10 +15,8 @@
 #ifndef _NOLIBC_ARCH_H
 #define _NOLIBC_ARCH_H
 
-#if defined(__x86_64__)
-#include "arch-x86_64.h"
-#elif defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__)
-#include "arch-i386.h"
+#if defined(__x86_64__) || defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__)
+#include "arch-x86.h"
 #elif defined(__ARM_EABI__)
 #include "arch-arm.h"
 #elif defined(__aarch64__)
Signature · 1 line
2.17.5
maintainer Acked-by on v1 1/3
#2 lore raw

Re: [RFC PATCH 1/3] tools/nolibc: merge i386 and x86_64 into a single x86 arch

Quoted 6 lines · Willy Tarreau

This remained the only exception to the kernel's architectures organization and it's always a bit cumbersome to deal with. Let's merge i386 and x86_64 into x86. This will result in a single arch-x86.h file by default, and we'll no longer need to merge the two manually during installation. Requesting either i386 or x86_64 will also result in installing x86.

We should do this in any case, independently from the series.

Quoted 9 lines
---
 tools/include/nolibc/Makefile                 |  10 +-
 tools/include/nolibc/arch-i386.h              | 178 -----------------
 .../nolibc/{arch-x86_64.h => arch-x86.h}      | 180 +++++++++++++++++-
 tools/include/nolibc/arch.h                   |   6 +-
 4 files changed, 179 insertions(+), 195 deletions(-)
 delete mode 100644 tools/include/nolibc/arch-i386.h
 rename tools/include/nolibc/{arch-x86_64.h => arch-x86.h} (53%)

<snip>

author maintainer
#3 lore raw

Re: [RFC PATCH 1/3] tools/nolibc: merge i386 and x86_64 into a single x86 arch

Quoted 9 lines · Thomas Weißschuh
Quoted 6 lines · Willy Tarreau

This remained the only exception to the kernel's architectures organization and it's always a bit cumbersome to deal with. Let's merge i386 and x86_64 into x86. This will result in a single arch-x86.h file by default, and we'll no longer need to merge the two manually during installation. Requesting either i386 or x86_64 will also result in installing x86.

We should do this in any case, independently from the series.

I agree, initially I made it part of the small improvements series but I thought it was less important.

Quoted 1 line

Thanks!
Willy

author maintainer
#4 lore raw

RFC PATCH 2/3 tools/nolibc: add a new target "headers_all_archs" to loop over all archs

In reply to Willy Tarreau

This target allows to install the nolibc headers for all supported architectures at once, just like it is in the development tree. This is a first step to support full multi-architecture support.

---
 tools/include/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/include/nolibc/Makefile +10 -0
index 9197c79b267a4..8de6ac5cec425 100644
@@ -23,6 +23,8 @@ else
 Q=@
 endif
 
+nolibc_supported_archs := aarch64 arm loongarch m68k mips powerpc riscv s390 sparc x86
+
 nolibc_arch := $(patsubst arm64,aarch64,$(ARCH))
 arch_file := arch-$(nolibc_arch).h
 all_files := \
@@ -83,6 +85,7 @@ help:
 	@echo "  all                 call \"headers\""
 	@echo "  clean               clean the sysroot"
 	@echo "  headers             prepare a sysroot in \$${OUTPUT}sysroot"
+	@echo "  headers_all_archs   prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
 	@echo ""
@@ -110,6 +113,13 @@ headers_standalone: headers
 	$(Q)$(MAKE) -C $(srctree) headers
 	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
 
+# installs headers for all archs at once.
+headers_all_archs:
+	$(Q)mkdir -p "$(OUTPUT)sysroot"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include"
+	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
+	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"
+
 # GCC uses "s390", clang "systemz"
 CLANG_CROSS_FLAGS := $(subst --target=s390-linux,--target=systemz-linux,$(CLANG_CROSS_FLAGS))
 
Signature · 1 line
2.17.5
#5 lore raw

Re: [RFC PATCH 2/3] tools/nolibc: add a new target "headers_all_archs" to loop over all archs

Quoted 31 lines · Willy Tarreau

This target allows to install the nolibc headers for all supported architectures at once, just like it is in the development tree. This is a first step to support full multi-architecture support.

---
 tools/include/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/include/nolibc/Makefile +3 -0
index 9197c79b267a4..8de6ac5cec425 100644
@@ -23,6 +23,8 @@ else
 Q=@
 endif
 
+nolibc_supported_archs := aarch64 arm loongarch m68k mips powerpc riscv s390 sparc x86
+
 nolibc_arch := $(patsubst arm64,aarch64,$(ARCH))
 arch_file := arch-$(nolibc_arch).h
 all_files := \
@@ -83,6 +85,7 @@ help:
 	@echo "  all                 call \"headers\""
 	@echo "  clean               clean the sysroot"
 	@echo "  headers             prepare a sysroot in \$${OUTPUT}sysroot"
+	@echo "  headers_all_archs   prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
 	@echo ""

tools/include/nolibc/Makefile headers_standalone: headers · line 121

Quoted 9 lines
tools/include/nolibc/Makefile +6 -0
@@ -110,6 +113,13 @@ headers_standalone: headers
 	$(Q)$(MAKE) -C $(srctree) headers
 	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
 
+# installs headers for all archs at once.
+headers_all_archs:
+	$(Q)mkdir -p "$(OUTPUT)sysroot"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include"
+	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
+	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"

IMO we could always just install all architecture headers. It's not much code after all.
If it is a problem for a user they can either just delete the superfluous architectures or do 'mv arch-$foo.h arch.h; rm arch-*.h'.

Quoted 6 lines
tools/include/nolibc/Makefile +1 -0
+
 # GCC uses "s390", clang "systemz"
 CLANG_CROSS_FLAGS := $(subst --target=s390-linux,--target=systemz-linux,$(CLANG_CROSS_FLAGS))
 
Signature · 1 line
2.17.5
author maintainer
#6 lore raw

Re: [RFC PATCH 2/3] tools/nolibc: add a new target "headers_all_archs" to loop over all archs

Hi Thomas,

Quoted 35 lines · Thomas Weißschuh
Quoted 34 lines · Willy Tarreau

This target allows to install the nolibc headers for all supported architectures at once, just like it is in the development tree. This is a first step to support full multi-architecture support.

---
 tools/include/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/include/nolibc/Makefile +3 -0
index 9197c79b267a4..8de6ac5cec425 100644
@@ -23,6 +23,8 @@ else
 Q=@
 endif
 
+nolibc_supported_archs := aarch64 arm loongarch m68k mips powerpc riscv s390 sparc x86
+
 nolibc_arch := $(patsubst arm64,aarch64,$(ARCH))
 arch_file := arch-$(nolibc_arch).h
 all_files := \
@@ -83,6 +85,7 @@ help:
 	@echo "  all                 call \"headers\""
 	@echo "  clean               clean the sysroot"
 	@echo "  headers             prepare a sysroot in \$${OUTPUT}sysroot"
+	@echo "  headers_all_archs   prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
 	@echo ""
@@ -110,6 +113,13 @@ headers_standalone: headers
 	$(Q)$(MAKE) -C $(srctree) headers
 	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
 
Quoted 11 lines

tools/include/nolibc/Makefile headers_standalone: headers · line 121

Quoted 6 lines
tools/include/nolibc/Makefile +6 -0
+# installs headers for all archs at once.
+headers_all_archs:
+	$(Q)mkdir -p "$(OUTPUT)sysroot"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include"
+	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
+	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"

IMO we could always just install all architecture headers. It's not much code after all.
If it is a problem for a user they can either just delete the superfluous architectures or do 'mv arch-$foo.h arch.h; rm arch-*.h'.

I wanted to do that first, then thought that maybe some would like to only install the nolibc headers because they already have the UAPI headers from another source (local libc, distro packages, toolchain etc). Even for us during nolibc development, not having to iterate through all archs to reinstall everything is a huge time saver.

However, I had another idea that floated in my mind, which is that given that we're only saving a few small arch-* files by not installing all archs all the time, maybe we should replace the "headers" target to always install nolibc headers for all archs like above, and keep the uapi headers install separate (only one or all). This would remove the need for the target above whose role is a bit ambiguous. What do you think ?

Thanks!
Willy

#7 lore raw

Re: [RFC PATCH 2/3] tools/nolibc: add a new target "headers_all_archs" to loop over all archs

Quoted 55 lines · Willy Tarreau
Quoted 35 lines · Thomas Weißschuh
Quoted 34 lines · Willy Tarreau

This target allows to install the nolibc headers for all supported architectures at once, just like it is in the development tree. This is a first step to support full multi-architecture support.

---
 tools/include/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/include/nolibc/Makefile +3 -0
index 9197c79b267a4..8de6ac5cec425 100644
@@ -23,6 +23,8 @@ else
 Q=@
 endif
 
+nolibc_supported_archs := aarch64 arm loongarch m68k mips powerpc riscv s390 sparc x86
+
 nolibc_arch := $(patsubst arm64,aarch64,$(ARCH))
 arch_file := arch-$(nolibc_arch).h
 all_files := \
@@ -83,6 +85,7 @@ help:
 	@echo "  all                 call \"headers\""
 	@echo "  clean               clean the sysroot"
 	@echo "  headers             prepare a sysroot in \$${OUTPUT}sysroot"
+	@echo "  headers_all_archs   prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
 	@echo ""
@@ -110,6 +113,13 @@ headers_standalone: headers
 	$(Q)$(MAKE) -C $(srctree) headers
 	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
 
Quoted 11 lines

tools/include/nolibc/Makefile headers_standalone: headers · line 121

Quoted 6 lines
tools/include/nolibc/Makefile +6 -0
+# installs headers for all archs at once.
+headers_all_archs:
+	$(Q)mkdir -p "$(OUTPUT)sysroot"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include"
+	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
+	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"

IMO we could always just install all architecture headers. It's not much code after all.
If it is a problem for a user they can either just delete the superfluous architectures or do 'mv arch-$foo.h arch.h; rm arch-*.h'.

I wanted to do that first, then thought that maybe some would like to only install the nolibc headers because they already have the UAPI headers from another source (local libc, distro packages, toolchain etc). Even for us during nolibc development, not having to iterate through all archs to reinstall everything is a huge time saver.

Quoted 7 lines

However, I had another idea that floated in my mind, which is that given that we're only saving a few small arch-* files by not installing all archs all the time, maybe we should replace the "headers" target to always install nolibc headers for all archs like above, and keep the uapi headers install separate (only one or all). This would remove the need for the target above whose role is a bit ambiguous. What do you think ?

That is exactly what I tried to express :-)

Thomas

author maintainer
#8 lore raw

Re: [RFC PATCH 2/3] tools/nolibc: add a new target "headers_all_archs" to loop over all archs

Quoted 64 lines · Thomas Weißschuh
Quoted 55 lines · Willy Tarreau
Quoted 35 lines · Thomas Weißschuh
Quoted 34 lines · Willy Tarreau

This target allows to install the nolibc headers for all supported architectures at once, just like it is in the development tree. This is a first step to support full multi-architecture support.

---
 tools/include/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/include/nolibc/Makefile +3 -0
index 9197c79b267a4..8de6ac5cec425 100644
@@ -23,6 +23,8 @@ else
 Q=@
 endif
 
+nolibc_supported_archs := aarch64 arm loongarch m68k mips powerpc riscv s390 sparc x86
+
 nolibc_arch := $(patsubst arm64,aarch64,$(ARCH))
 arch_file := arch-$(nolibc_arch).h
 all_files := \
@@ -83,6 +85,7 @@ help:
 	@echo "  all                 call \"headers\""
 	@echo "  clean               clean the sysroot"
 	@echo "  headers             prepare a sysroot in \$${OUTPUT}sysroot"
+	@echo "  headers_all_archs   prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
 	@echo ""
@@ -110,6 +113,13 @@ headers_standalone: headers
 	$(Q)$(MAKE) -C $(srctree) headers
 	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)sysroot
 
Quoted 11 lines

tools/include/nolibc/Makefile headers_standalone: headers · line 121

Quoted 6 lines
tools/include/nolibc/Makefile +6 -0
+# installs headers for all archs at once.
+headers_all_archs:
+	$(Q)mkdir -p "$(OUTPUT)sysroot"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include"
+	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
+	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"

IMO we could always just install all architecture headers. It's not much code after all.
If it is a problem for a user they can either just delete the superfluous architectures or do 'mv arch-$foo.h arch.h; rm arch-*.h'.

I wanted to do that first, then thought that maybe some would like to only install the nolibc headers because they already have the UAPI headers from another source (local libc, distro packages, toolchain etc). Even for us during nolibc development, not having to iterate through all archs to reinstall everything is a huge time saver.

Quoted 7 lines

However, I had another idea that floated in my mind, which is that given that we're only saving a few small arch-* files by not installing all archs all the time, maybe we should replace the "headers" target to always install nolibc headers for all archs like above, and keep the uapi headers install separate (only one or all). This would remove the need for the target above whose role is a bit ambiguous. What do you think ?

Quoted 1 line

That is exactly what I tried to express :-)

Ah OK, I understood that you wanted to install all UAPI in the target above :-) OK that's fine then, will do that.

Willy

author maintainer
#9 lore raw

RFC PATCH 3/3 tools/nolibc: add a new "install_all_archs" target

In reply to Willy Tarreau

This installs all supported archs together, both from nolibc and kernel headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch, and asm/ is rebuilt from all these files in order to include the right one depending on the build architecture.

This allows to use a single unified sysroot for all archs, and to only change the compiler or the target architecture. This way, a complete sysroot is much easier to use (a single directory is needed) and much smaller.

---
 tools/include/nolibc/Makefile | 39 +++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
tools/include/nolibc/Makefile +39 -0
index 8de6ac5cec425..b6ed11d0b5419 100644
@@ -88,6 +88,7 @@ help:
 	@echo "  headers_all_archs   prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
+	@echo "  install_all_archs   install a multi-arch sysroot + kernel headers in \$${OUTPUT}sysroot"
 	@echo ""
 	@echo "These targets may also be called from tools as \"make nolibc_<target>\"."
 	@echo ""
@@ -120,6 +121,44 @@ headers_all_archs:
 	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
 	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"
 
+install_all_archs: headers_all_archs
+	@# install common headers for any arch, take them all. This will clear everything.
+	$(Q)$(MAKE) -C $(srctree) ARCH=x86 mrproper
+	$(Q)$(MAKE) -C $(srctree) ARCH=x86 headers_install no-export-headers= INSTALL_HDR_PATH="$(OUTPUT)sysroot"
+	@# remove the contents of the unused asm dir which we will rebuild from the arch ones
+	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
+	@# Now install headers for all archs
+	$(Q)for arch in $(patsubst aarch64,arm64,$(nolibc_supported_archs)); do \
+		echo "# installing $$arch"; \
+		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \
+			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \
+			$(MAKE) -C $(srctree) ARCH=$$arch headers_install no-export-headers= \
+				INSTALL_HDR_PATH="$(OUTPUT)sysroot/include/$$arch" >/dev/null; \
+			mv "$(OUTPUT)sysroot/include/$$arch/include/asm" "$(OUTPUT)sysroot/include/asm-arch-$$arch"; \
+			rm -rf "$(OUTPUT)sysroot/include/$$arch"; \
+			case "$$arch" in \
+				arm)       cond="defined(__ARM_EABI__)" ;; \
+				arm64)     cond="defined(__aarch64__)" ;; \
+				loongarch) cond="defined(__loongarch__)" ;; \
+				m68k)      cond="defined(__m68k__)" ;; \
+				mips)      cond="defined(__mips__)" ;; \
+				powerpc)   cond="defined(__powerpc__)" ;; \
+				riscv)     cond="defined(__riscv__)" ;; \
+				s390)      cond="defined(__s390__) || defined(__s390x__)" ;; \
+				sparc)     cond="defined(__sparc__)" ;; \
+				x86)       cond="defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__) || defined(__x86_64__)";; \
+				*)         echo "Unsupported arch" >&2; exit 1;; \
+			esac;\
+			for file in "$(OUTPUT)sysroot/include/asm-arch-$$arch/"*.h; do \
+				base="$${file##*/}"; \
+				( echo "#if $$cond"; \
+				  echo "#include \"../asm-arch-$$arch/$$base\""; \
+				  echo "#endif" ) >> "$(OUTPUT)sysroot/include/asm/$$base"; \
+			done; \
+		fi;\
+	done
+
 # GCC uses "s390", clang "systemz"
 CLANG_CROSS_FLAGS := $(subst --target=s390-linux,--target=systemz-linux,$(CLANG_CROSS_FLAGS))
 
Signature · 1 line
2.17.5
#10 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 56 lines · Willy Tarreau

This installs all supported archs together, both from nolibc and kernel headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch, and asm/ is rebuilt from all these files in order to include the right one depending on the build architecture.

This allows to use a single unified sysroot for all archs, and to only change the compiler or the target architecture. This way, a complete sysroot is much easier to use (a single directory is needed) and much smaller.

---
 tools/include/nolibc/Makefile | 39 +++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
tools/include/nolibc/Makefile +26 -0
index 8de6ac5cec425..b6ed11d0b5419 100644
@@ -88,6 +88,7 @@ help:
 	@echo "  headers_all_archs   prepare a multi-arch sysroot in \$${OUTPUT}sysroot"
 	@echo "  headers_standalone  like \"headers\", and also install kernel headers"
 	@echo "  help                this help"
+	@echo "  install_all_archs   install a multi-arch sysroot + kernel headers in \$${OUTPUT}sysroot"
 	@echo ""
 	@echo "These targets may also be called from tools as \"make nolibc_<target>\"."
 	@echo ""
@@ -120,6 +121,44 @@ headers_all_archs:
 	$(Q)cp --parents $(all_files) arch.h "$(OUTPUT)sysroot/include/"
 	$(Q)cp $(addsuffix .h,$(addprefix arch-,$(nolibc_supported_archs))) "$(OUTPUT)sysroot/include/"
 
+install_all_archs: headers_all_archs
+	@# install common headers for any arch, take them all. This will clear everything.
+	$(Q)$(MAKE) -C $(srctree) ARCH=x86 mrproper
+	$(Q)$(MAKE) -C $(srctree) ARCH=x86 headers_install no-export-headers= INSTALL_HDR_PATH="$(OUTPUT)sysroot"
+	@# remove the contents of the unused asm dir which we will rebuild from the arch ones
+	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
+	@# Now install headers for all archs
+	$(Q)for arch in $(patsubst aarch64,arm64,$(nolibc_supported_archs)); do \
+		echo "# installing $$arch"; \
+		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \
+			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \
+			$(MAKE) -C $(srctree) ARCH=$$arch headers_install no-export-headers= \
+				INSTALL_HDR_PATH="$(OUTPUT)sysroot/include/$$arch" >/dev/null; \
+			mv "$(OUTPUT)sysroot/include/$$arch/include/asm" "$(OUTPUT)sysroot/include/asm-arch-$$arch"; \
+			rm -rf "$(OUTPUT)sysroot/include/$$arch"; \
+			case "$$arch" in \
+				arm)       cond="defined(__ARM_EABI__)" ;; \
+				arm64)     cond="defined(__aarch64__)" ;; \
+				loongarch) cond="defined(__loongarch__)" ;; \
+				m68k)      cond="defined(__m68k__)" ;; \
+				mips)      cond="defined(__mips__)" ;; \
+				powerpc)   cond="defined(__powerpc__)" ;; \
+				riscv)     cond="defined(__riscv__)" ;; \
+				s390)      cond="defined(__s390__) || defined(__s390x__)" ;; \

tools/include/nolibc/Makefile headers_all_archs: · line 158

Quoted 10 lines
tools/include/nolibc/Makefile +10 -0
+				sparc)     cond="defined(__sparc__)" ;; \
+				x86)       cond="defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__) || defined(__x86_64__)";; \
+				*)         echo "Unsupported arch" >&2; exit 1;; \
+			esac;\
+			for file in "$(OUTPUT)sysroot/include/asm-arch-$$arch/"*.h; do \
+				base="$${file##*/}"; \
+				( echo "#if $$cond"; \
+				  echo "#include \"../asm-arch-$$arch/$$base\""; \
+				  echo "#endif" ) >> "$(OUTPUT)sysroot/include/asm/$$base"; \
+			done; \

I'm not a fan of the loop to build the ifdeffery. It is a duplication of what we have in tools/include/nolibc/arch.h and horrible to look at. Can we stick this into a reusable header file?
Something along the lines of this:

	/* asm/foo.h */
	#define _NOLIBC_PER_ARCH_HEADER "foo.h"
	#include "_nolibc_include_per_arch_header.h"


	/* _nolibc_include_per_arch_header.h */
	#if defined(__i386__)
	#include CONCAT("asm-arch-x86/", _NOLIBC_PER_ARCH_HEADER)
	#elif
	...

However, so far I couldn't get it to work.
Also it would be great if we can use it for the current arch.h, too.

Quoted 8 lines
tools/include/nolibc/Makefile +3 -0
+		fi;\
+	done
+
 # GCC uses "s390", clang "systemz"
 CLANG_CROSS_FLAGS := $(subst --target=s390-linux,--target=systemz-linux,$(CLANG_CROSS_FLAGS))
 
Signature · 1 line
2.17.5
#11 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 38 lines · Thomas Weißschuh
Quoted 10 lines · Willy Tarreau

This installs all supported archs together, both from nolibc and kernel headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch, and asm/ is rebuilt from all these files in order to include the right one depending on the build architecture.

This allows to use a single unified sysroot for all archs, and to only change the compiler or the target architecture. This way, a complete sysroot is much easier to use (a single directory is needed) and much smaller.

tools/include/nolibc/Makefile headers_all_archs: · line 137

Quoted 9 lines
tools/include/nolibc/Makefile +9 -0
+	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
+	@# Now install headers for all archs
+	$(Q)for arch in $(patsubst aarch64,arm64,$(nolibc_supported_archs)); do \
+		echo "# installing $$arch"; \
+		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \
+			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \
+			$(MAKE) -C $(srctree) ARCH=$$arch headers_install no-export-headers= \
+				INSTALL_HDR_PATH="$(OUTPUT)sysroot/include/$$arch" >/dev/null; \

I'm not a fan of the loop to build the ifdeffery. It is a duplication of what we have in tools/include/nolibc/arch.h and horrible to look at. Can we stick this into a reusable header file?
Something along the lines of this:

	/* asm/foo.h */
	#define _NOLIBC_PER_ARCH_HEADER "foo.h"
	#include "_nolibc_include_per_arch_header.h"


	/* _nolibc_include_per_arch_header.h */
	#if defined(__i386__)
	#include CONCAT("asm-arch-x86/", _NOLIBC_PER_ARCH_HEADER)
	#elif
	...
Quoted 2 lines

However, so far I couldn't get it to work.
Also it would be great if we can use it for the current arch.h, too.

I'm not sure either of those is better than the version we
had until commit f3c8d4c7a728 ("kbuild: remove headers_{install,check}_all"). which simply relied on a symlink to the architecture specific directory to be set.

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

Arnd

author maintainer
#12 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 47 lines · Arnd Bergmann
Quoted 38 lines · Thomas Weißschuh
Quoted 10 lines · Willy Tarreau

This installs all supported archs together, both from nolibc and kernel headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch, and asm/ is rebuilt from all these files in order to include the right one depending on the build architecture.

This allows to use a single unified sysroot for all archs, and to only change the compiler or the target architecture. This way, a complete sysroot is much easier to use (a single directory is needed) and much smaller.

tools/include/nolibc/Makefile headers_all_archs: · line 137

Quoted 9 lines
tools/include/nolibc/Makefile +9 -0
+	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
+	@# Now install headers for all archs
+	$(Q)for arch in $(patsubst aarch64,arm64,$(nolibc_supported_archs)); do \
+		echo "# installing $$arch"; \
+		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \
+			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \
+			$(MAKE) -C $(srctree) ARCH=$$arch headers_install no-export-headers= \
+				INSTALL_HDR_PATH="$(OUTPUT)sysroot/include/$$arch" >/dev/null; \

I'm not a fan of the loop to build the ifdeffery. It is a duplication of what we have in tools/include/nolibc/arch.h and horrible to look at. Can we stick this into a reusable header file?
Something along the lines of this:

	/* asm/foo.h */
	#define _NOLIBC_PER_ARCH_HEADER "foo.h"
	#include "_nolibc_include_per_arch_header.h"


	/* _nolibc_include_per_arch_header.h */
	#if defined(__i386__)
	#include CONCAT("asm-arch-x86/", _NOLIBC_PER_ARCH_HEADER)
	#elif
	...
Quoted 2 lines

However, so far I couldn't get it to work.
Also it would be great if we can use it for the current arch.h, too.

I'm not sure either of those is better than the version we
had until commit f3c8d4c7a728 ("kbuild: remove headers_{install,check}_all"). which simply relied on a symlink to the architecture specific directory to be set.

Quoted 6 lines

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

I have never found how it would be possible to do that, let alone in a more or less portable way, because #include doesn't take a C-string in argument but a special syntax which is specific to it ("x" or <x>). It doesn't support concatenating strings for example:

  #include "stdio"".h"
  $ gcc -E inc.h
  inc.h:1:17: warning: extra tokens at end of #include directive
      1 | #include "stdio"".h"
        |                 ^~~~
  inc.h:1:10: fatal error: stdio: No such file or directory
      1 | #include "stdio"".h"
        |          ^~~~~~~

Willy

#13 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 17 lines · Willy Tarreau
Quoted 8 lines · Arnd Bergmann

On Thu, Jun 26, 2025, at 22:18, Thomas Weißschuh wrote:

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

I have never found how it would be possible to do that, let alone in a more or less portable way, because #include doesn't take a C-string in argument but a special syntax which is specific to it ("x" or <x>). It doesn't support concatenating strings for example:

  #include "stdio"".h"
Quoted 7 lines
  $ gcc -E inc.h
  inc.h:1:17: warning: extra tokens at end of #include directive
      1 | #include "stdio"".h"
        |                 ^~~~
  inc.h:1:10: fatal error: stdio: No such file or directory
      1 | #include "stdio"".h"
        |          ^~~~~~~

This variant does seem to work with gcc and clang:

#define ARCH i386
#define __ARCH_HEADER(a, b) <a ## _ ## b>
#define _ARCH_HEADER(a, b) __ARCH_HEADER(a, b)
#include _ARCH_HEADER(ARCH, stdio.h)

Or even simpler

#define _NOLIBC_ARCH_HEADER(file) <i386_ ## file>

The ## concatenation is a bit limited here since cpp requires both sides to expand to an identifier, meaning you can have a '_' next to it, but not a '/' or '-'.

Arnd

author maintainer
#14 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 33 lines · Arnd Bergmann
Quoted 17 lines · Willy Tarreau
Quoted 8 lines · Arnd Bergmann

On Thu, Jun 26, 2025, at 22:18, Thomas Weißschuh wrote:

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

I have never found how it would be possible to do that, let alone in a more or less portable way, because #include doesn't take a C-string in argument but a special syntax which is specific to it ("x" or <x>). It doesn't support concatenating strings for example:

  #include "stdio"".h"
Quoted 7 lines
  $ gcc -E inc.h
  inc.h:1:17: warning: extra tokens at end of #include directive
      1 | #include "stdio"".h"
        |                 ^~~~
  inc.h:1:10: fatal error: stdio: No such file or directory
      1 | #include "stdio"".h"
        |          ^~~~~~~

This variant does seem to work with gcc and clang:

#define ARCH i386
#define __ARCH_HEADER(a, b) <a ## _ ## b>
#define _ARCH_HEADER(a, b) __ARCH_HEADER(a, b)
#include _ARCH_HEADER(ARCH, stdio.h)

Quoted 7 lines

Or even simpler

#define _NOLIBC_ARCH_HEADER(file) <i386_ ## file>

The ## concatenation is a bit limited here since cpp requires both sides to expand to an identifier, meaning you can have a '_' next to it, but not a '/' or '-'.

Thanks Arnd, will give it a try this week-end.

Willy

#15 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

In reply to Arnd Bergmann
Quoted 39 lines · Arnd Bergmann
Quoted 38 lines · Thomas Weißschuh
Quoted 10 lines · Willy Tarreau

This installs all supported archs together, both from nolibc and kernel headers. The arch-specific asm/ subdirs are renamed to asm-arch-$arch, and asm/ is rebuilt from all these files in order to include the right one depending on the build architecture.

This allows to use a single unified sysroot for all archs, and to only change the compiler or the target architecture. This way, a complete sysroot is much easier to use (a single directory is needed) and much smaller.

tools/include/nolibc/Makefile headers_all_archs: · line 137

Quoted 9 lines
tools/include/nolibc/Makefile +9 -0
+	$(Q)rm -rf "$(OUTPUT)sysroot/include/asm"
+	$(Q)mkdir -p "$(OUTPUT)sysroot/include/asm"
+	@# Now install headers for all archs
+	$(Q)for arch in $(patsubst aarch64,arm64,$(nolibc_supported_archs)); do \
+		echo "# installing $$arch"; \
+		if ! [ -d $(OUTPUT)sysroot/include/asm-arch-$$arch ]; then \
+			$(MAKE) -C $(srctree) ARCH=$$arch mrproper; \
+			$(MAKE) -C $(srctree) ARCH=$$arch headers_install no-export-headers= \
+				INSTALL_HDR_PATH="$(OUTPUT)sysroot/include/$$arch" >/dev/null; \

I'm not a fan of the loop to build the ifdeffery. It is a duplication of what we have in tools/include/nolibc/arch.h and horrible to look at. Can we stick this into a reusable header file?
Something along the lines of this:

	/* asm/foo.h */
	#define _NOLIBC_PER_ARCH_HEADER "foo.h"
	#include "_nolibc_include_per_arch_header.h"


	/* _nolibc_include_per_arch_header.h */
	#if defined(__i386__)
	#include CONCAT("asm-arch-x86/", _NOLIBC_PER_ARCH_HEADER)
	#elif
	...
Quoted 7 lines
Quoted 2 lines

However, so far I couldn't get it to work.
Also it would be great if we can use it for the current arch.h, too.

I'm not sure either of those is better than the version we
had until commit f3c8d4c7a728 ("kbuild: remove headers_{install,check}_all"). which simply relied on a symlink to the architecture specific directory to be set.

Thanks for the pointer, that probably answers the question if this could be part of kbuild proper. It shouldn't.

With the symlink, a given generic UAPI tree can be specialized to one specific architecture. But here we want to create a full sysroot that works for all architectures *at the same time*. So a symlink would not be enough.

Quoted 6 lines

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

If we can't get it to work like this I would still prefer to have a template header file which gets specialized with sed instead of the Makefile loop.

Thomas

author maintainer
#16 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 1 line · Thomas Weißschuh

On 2025-06-26 23:15:07+0200, Arnd Bergmann wrote:

(...)

Quoted 3 lines

With the symlink, a given generic UAPI tree can be specialized to one specific architecture. But here we want to create a full sysroot that works for all architectures *at the same time*. So a symlink would not be enough.

Exactly!

Quoted 10 lines
Quoted 6 lines

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

If we can't get it to work like this I would still prefer to have a template header file which gets specialized with sed instead of the Makefile loop.

The thing is that it's not a single header, it's for each header file present in asm/. And we can't request that anyone adding anything into asm would also have to maintain one extra template for each of them.

Or I'm simply not getting how you would envision it maybe.

Willy

author maintainer
#17 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 20 lines · Willy Tarreau
Quoted 1 line · Thomas Weißschuh

On 2025-06-26 23:15:07+0200, Arnd Bergmann wrote:

(...)

Quoted 3 lines

With the symlink, a given generic UAPI tree can be specialized to one specific architecture. But here we want to create a full sysroot that works for all architectures *at the same time*. So a symlink would not be enough.

Exactly!

Quoted 10 lines
Quoted 6 lines

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

If we can't get it to work like this I would still prefer to have a template header file which gets specialized with sed instead of the Makefile loop.

Quoted 5 lines

The thing is that it's not a single header, it's for each header file present in asm/. And we can't request that anyone adding anything into asm would also have to maintain one extra template for each of them.

Or I'm simply not getting how you would envision it maybe.

Or do you mean a template that contains all #ifdef/#include for all archs, that serves as the basis to rebuild all headers, and that we still have the loop on all files in the makefile ? I.e. you simply want to drop the arch loop ? If that's it, yes I'm fine with this as well and can give it a try.

Willy

#18 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 25 lines · Willy Tarreau
Quoted 24 lines · Willy Tarreau
Quoted 1 line · Thomas Weißschuh

On 2025-06-26 23:15:07+0200, Arnd Bergmann wrote:

(...)

Quoted 3 lines

With the symlink, a given generic UAPI tree can be specialized to one specific architecture. But here we want to create a full sysroot that works for all architectures *at the same time*. So a symlink would not be enough.

Exactly!

Quoted 10 lines
Quoted 6 lines

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

If we can't get it to work like this I would still prefer to have a template header file which gets specialized with sed instead of the Makefile loop.

The thing is that it's not a single header, it's for each header file present in asm/. And we can't request that anyone adding anything into asm would also have to maintain one extra template for each of them.

Quoted 7 lines
Quoted 1 line

Or I'm simply not getting how you would envision it maybe.

Or do you mean a template that contains all #ifdef/#include for all archs, that serves as the basis to rebuild all headers, and that we still have the loop on all files in the makefile ? I.e. you simply want to drop the arch loop ? If that's it, yes I'm fine with this as well and can give it a try.

This is exactly what I meant.

Thomas

author maintainer
#19 lore raw

Re: [RFC PATCH 3/3] tools/nolibc: add a new "install_all_archs" target

Quoted 28 lines · Thomas Weißschuh
Quoted 27 lines · Willy Tarreau
Quoted 25 lines · Willy Tarreau
Quoted 1 line · Thomas Weißschuh

On 2025-06-26 23:15:07+0200, Arnd Bergmann wrote:

(...)

Quoted 3 lines

With the symlink, a given generic UAPI tree can be specialized to one specific architecture. But here we want to create a full sysroot that works for all architectures *at the same time*. So a symlink would not be enough.

Exactly!

Quoted 10 lines
Quoted 6 lines

If it's indeed possible to concatenate the path name (I couldn't figure that out either), that could also be done in place of the symlink but simpler than the #if/#elif/#elif/... block, like

#include <arch.h> // defines ARCH_PREFIX
#include CONCAT(ARCH_PREFIX, ioctl.h)

If we can't get it to work like this I would still prefer to have a template header file which gets specialized with sed instead of the Makefile loop.

The thing is that it's not a single header, it's for each header file present in asm/. And we can't request that anyone adding anything into asm would also have to maintain one extra template for each of them.

Or I'm simply not getting how you would envision it maybe.

Quoted 7 lines
Quoted 5 lines

Or do you mean a template that contains all #ifdef/#include for all archs, that serves as the basis to rebuild all headers, and that we still have the loop on all files in the makefile ? I.e. you simply want to drop the arch loop ? If that's it, yes I'm fine with this as well and can give it a try.

This is exactly what I meant.

OK then, I'll have a look to this and Arnd's idea, hopefully we'll find something better than the current proposal.

willy