Skip to contentSkip to conversation

check-uapi: improve portability for testing headers

PATCH In review Arnd Bergmann 5 participants 12 messages Mar 6 – Mar 20

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover check-uapi: improve portability for testing headers

From: Arnd Bergmann <[email protected]>

While working on a series to clean up some uapi headers, I needed to check the that the actual ABI remains unchanged. I found that scripts/check-uapi.sh works well enough for architectures that have a full toolchain installed, but not with the nolibc compilers I provide on kernel.org.

I ended up with a series addressing three separate issues here, but in the end I can now validate ABI changes across all supported architectures and ABIs.

The third patch depends on a series from Thomas Weißschuh that was just merged into the kbuild-for-next tree, the other ones could also apply to older kernels. I have marked the second patch for backports to stable kernels, this one is what caused me the most work in debugging.

Arnd

Arnd Bergmann (3):
check-uapi: link into shared objects
check-uapi: honor ${CROSS_COMPILE} setting
check-uapi: use dummy libc includes

 scripts/check-uapi.sh | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)
Signature · 8 lines
2.39.5
Cc: Dodji Seketeli <[email protected]>
Cc: John Moon <[email protected]>
Cc: Nathan Chancellor <[email protected]>
Cc: Nicolas Schier <[email protected]>
Cc: Thomas Weißschuh <[email protected]>
Cc: [email protected]
Cc: [email protected]
#1 lore raw

PATCH 1/3 check-uapi: link into shared objects

From: Arnd Bergmann <[email protected]>

While testing ABI changes across all architectures, I found that abidiff sometimes produces nonsensical output. Further debugging identified missing or broken libelf support for architecture specific relocations in ET_REL binaries as the source of the problem[1].

Change the script to no longer produce a relocatable object file but instead create a shared library for each header. This makes abidiff work for all of the architectures in upstream linux kernels.

---
 scripts/check-uapi.sh | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
scripts/check-uapi.sh +5 -2
index 955581735cb3..9fa45cbdecc2 100755
@@ -178,8 +178,11 @@ do_compile() {
 	local -r inc_dir="$1"
 	local -r header="$2"
 	local -r out="$3"
-	printf "int main(void) { return 0; }\n" | \
-		"$CC" -c \
+	printf "int f(void) { return 0; }\n" | \
+		"$CC" \
+		  -shared \
+		  -nostdlib \
+		  -fPIC \
 		  -o "$out" \
 		  -x c \
 		  -O0 \
Signature · 1 line
2.39.5
#2 lore raw

PATCH 2/3 check-uapi: honor ${CROSS_COMPILE} setting

In reply to Arnd Bergmann

From: Arnd Bergmann <[email protected]>

When ${CROSS_COMPILE} is set, but ${CC} is not set, the logic in check-uapi.sh is different from the top-level Makefile, which defaults to using the cross gcc. This leads to using the native gcc instead of the cross version, resulting in unexpected false-positive and false-negative output.

Use the same logic here that we use in Kbuild for consistency.

---
 scripts/check-uapi.sh | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
scripts/check-uapi.sh +6 -5
index 9fa45cbdecc2..e4d120eb09e3 100755
@@ -33,9 +33,10 @@ Options:
     -v             Verbose operation (print more information about each header being checked).
 
 Environmental args:
-    ABIDIFF  Custom path to abidiff binary
-    CC       C compiler (default is "gcc")
-    ARCH     Target architecture for the UAPI check (default is host arch)
+    ABIDIFF        Custom path to abidiff binary
+    CROSS_COMPILE  Toolchain prefix for compiler
+    CC             C compiler (default is "\${CROSS_COMPILE}gcc")
+    ARCH           Target architecture for the UAPI check (default is host arch)
 
 Exit codes:
     $SUCCESS) Success
@@ -198,7 +199,7 @@ do_compile() {
 run_make_headers_install() {
 	local -r ref="$1"
 	local -r install_dir="$(get_header_tree "$ref")"
-	make -j "$MAX_THREADS" ARCH="$ARCH" INSTALL_HDR_PATH="$install_dir" \
+	make -j "$MAX_THREADS" CROSS_COMPILE="${CROSS_COMPILE}" ARCH="$ARCH" INSTALL_HDR_PATH="$install_dir" \
 		headers_install > /dev/null
 }
 
@@ -407,7 +408,7 @@ min_version_is_satisfied() {
 # Make sure we have the tools we need and the arguments make sense
 check_deps() {
 	ABIDIFF="${ABIDIFF:-abidiff}"
-	CC="${CC:-gcc}"
+	CC="${CC:-${CROSS_COMPILE}gcc}"
 	ARCH="${ARCH:-$(uname -m)}"
 	if [ "$ARCH" = "x86_64" ]; then
 		ARCH="x86"
Signature · 1 line
2.39.5
#3 lore raw

PATCH 3/3 check-uapi: use dummy libc includes

In reply to Arnd Bergmann

From: Arnd Bergmann <[email protected]>

Based on Thomas Weißschuh's series to kernel headers to no longer require an installed libc when build testing the uapi headers, the same can now be done for the scripts/check-uapi.sh script.

The only required change here is to add the usr/dummy-include include path.

---
 scripts/check-uapi.sh | 1 +
 1 file changed, 1 insertion(+)
scripts/check-uapi.sh +1 -0
index e4d120eb09e3..c8beec58871c 100755
@@ -191,6 +191,7 @@ do_compile() {
 		  -fno-eliminate-unused-debug-types \
 		  -g \
 		  "-I${inc_dir}" \
+		  "-Iusr/dummy-include" \
 		  -include "$header" \
 		  -
 }
Signature · 1 line
2.39.5
#4 lore raw

Re: [PATCH 3/3] check-uapi: use dummy libc includes

Quoted 20 lines · Arnd Bergmann

From: Arnd Bergmann <[email protected]>

Based on Thomas Weißschuh's series to kernel headers to no longer require an installed libc when build testing the uapi headers, the same can now be done for the scripts/check-uapi.sh script.

The only required change here is to add the usr/dummy-include include path.

---
 scripts/check-uapi.sh | 1 +
 1 file changed, 1 insertion(+)
scripts/check-uapi.sh +0 -0
index e4d120eb09e3..c8beec58871c 100755

scripts/check-uapi.sh do_compile() { · line 194

Quoted 4 lines
scripts/check-uapi.sh +1 -0
@@ -191,6 +191,7 @@ do_compile() {
 		  -fno-eliminate-unused-debug-types \
 		  -g \
 		  "-I${inc_dir}" \
+		  "-Iusr/dummy-include" \

What about also using -nostdinc?

scripts/check-uapi.sh do_compile() { · line 197

Quoted 3 lines
scripts/check-uapi.sh +0 -0
 		  -include "$header" \
 		  -
 }

I have a similar (unfinished) patch flying around which also uses usr/dummy-include from the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.

scripts/check-uapi.sh +3 -0
index 955581735cb3..48d157ee5408 100755
@@ -186,7 +186,9 @@ do_compile() {
                  -std=c90 \
                  -fno-eliminate-unused-debug-types \
                  -g \
+                 -nostdinc \
                  "-I${inc_dir}" \
+                 "-I${inc_dir}/../dummy-include" \
                  -include "$header" \
                  -
 }
@@ -197,6 +199,7 @@ run_make_headers_install() {
        local -r install_dir="$(get_header_tree "$ref")"
        make -j "$MAX_THREADS" ARCH="$ARCH" INSTALL_HDR_PATH="$install_dir" \
                headers_install > /dev/null
+       rsync -a usr/dummy-include/ "$install_dir"/dummy-include/
 }
 
 # Install headers for both git refs
#5 lore raw

Re: [PATCH 3/3] check-uapi: use dummy libc includes

Quoted 6 lines · Thomas Weißschuh

scripts/check-uapi.sh do_compile() {

Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1 -0
 		  -g \
 		  "-I${inc_dir}" \
+		  "-Iusr/dummy-include" \

What about also using -nostdinc?

I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.

Quoted 4 lines

I have a similar (unfinished) patch flying around which also uses usr/dummy-include from the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.

Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.

Arnd

#6 lore raw

Re: [PATCH 3/3] check-uapi: use dummy libc includes

Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh

scripts/check-uapi.sh do_compile() {

Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1 -0
 		  -g \
 		  "-I${inc_dir}" \
+		  "-Iusr/dummy-include" \

What about also using -nostdinc?

I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.

Ack, your choice.

Quoted 9 lines
Quoted 4 lines

I have a similar (unfinished) patch flying around which also uses usr/dummy-include from the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.

Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.

It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.

Thomas

maintainer
#7 lore raw

Re: [PATCH 3/3] check-uapi: use dummy libc includes

Hi Arnd and Thomas,

Quoted 24 lines · Thomas Weißschuh
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh

scripts/check-uapi.sh do_compile() {

Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1 -0
 		  -g \
 		  "-I${inc_dir}" \
+		  "-Iusr/dummy-include" \

What about also using -nostdinc?

I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.

Ack, your choice.

Quoted 9 lines
Quoted 4 lines

I have a similar (unfinished) patch flying around which also uses usr/dummy-include from the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.

Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.

Quoted 7 lines

It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.

Thomas

Shall I wait some more days for a possible modified patch 3/3? Otherwise I'd like to apply the series to kbuild-next-unstable.

Thanks and kind regards,
Nicolas

Reviewed-by
#8 lore raw

Re: [PATCH 3/3] check-uapi: use dummy libc includes

Quoted 25 lines · Nicolas Schier
Quoted 24 lines · Thomas Weißschuh
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh

scripts/check-uapi.sh do_compile() {

Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1 -0
 		  -g \
 		  "-I${inc_dir}" \
+		  "-Iusr/dummy-include" \

What about also using -nostdinc?

I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.

Ack, your choice.

Quoted 9 lines
Quoted 4 lines

I have a similar (unfinished) patch flying around which also uses usr/dummy-include from the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.

Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.

Quoted 7 lines
Quoted 4 lines

It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.

Shall I wait some more days for a possible modified patch 3/3? Otherwise I'd like to apply the series to kbuild-next-unstable.

It is a somewhat theoretical issue and if it arises at some point we can still fix it up. So this shouldn't block the patches.

For the series:

Thomas

maintainer
#9 lore raw

Re: [PATCH 3/3] check-uapi: use dummy libc includes

Quoted 37 lines · Thomas Weißschuh
Quoted 25 lines · Nicolas Schier
Quoted 24 lines · Thomas Weißschuh
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh

scripts/check-uapi.sh do_compile() {

Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1 -0
 		  -g \
 		  "-I${inc_dir}" \
+		  "-Iusr/dummy-include" \

What about also using -nostdinc?

I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.

Ack, your choice.

Quoted 9 lines
Quoted 4 lines

I have a similar (unfinished) patch flying around which also uses usr/dummy-include from the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.

Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.

Quoted 7 lines
Quoted 4 lines

It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.

Shall I wait some more days for a possible modified patch 3/3? Otherwise I'd like to apply the series to kbuild-next-unstable.

It is a somewhat theoretical issue and if it arises at some point we can still fix it up. So this shouldn't block the patches.

Quoted 6 lines

For the series:

Thomas

ok, thanks!

Signature · 1 line
Nicolas
maintainer Acked-by on the v1 series
#10 lore raw
In reply to Arnd Bergmann
Quoted 18 lines · Arnd Bergmann

From: Arnd Bergmann <[email protected]>

While working on a series to clean up some uapi headers, I needed to check the that the actual ABI remains unchanged. I found that scripts/check-uapi.sh works well enough for architectures that have a full toolchain installed, but not with the nolibc compilers I provide on kernel.org.

I ended up with a series addressing three separate issues here, but in the end I can now validate ABI changes across all supported architectures and ABIs.

The third patch depends on a series from Thomas Weißschuh that was just merged into the kbuild-for-next tree, the other ones could also apply to older kernels. I have marked the second patch for backports to stable kernels, this one is what caused me the most work in debugging.

Quoted 6 lines

Arnd

Arnd Bergmann (3):
check-uapi: link into shared objects
check-uapi: honor ${CROSS_COMPILE} setting
check-uapi: use dummy libc includes

I assume that the shared objects change does not impact catching ABI differences?

Cheers,
Nathan

maintainer
#11 lore raw
In reply to Arnd Bergmann
Quoted 9 lines · Arnd Bergmann

From: Arnd Bergmann <[email protected]>

While working on a series to clean up some uapi headers, I needed to check the that the actual ABI remains unchanged. I found that scripts/check-uapi.sh works well enough for architectures that have a full toolchain installed, but not with the nolibc compilers I provide on kernel.org.

[...]

Applied to kbuild/linux.git (kbuild-next-unstable), thanks!

[1/3] check-uapi: link into shared objects

      https://git.kernel.org/kbuild/c/a261f6df
[2/3] check-uapi: honor ${CROSS_COMPILE} setting
      https://git.kernel.org/kbuild/c/9940ec38
[3/3] check-uapi: use dummy libc includes
      https://git.kernel.org/kbuild/c/bb25b563

Please look out for regression or issue reports or other follow up comments, as they may result in the patch/series getting dropped, reverted or modified (e.g. trailers). Patches applied to the kbuild-next-unstable branch are accepted pending wider testing in linux-next and any post-commit review; they will generally be moved to the kbuild-next branch in about a week if no issues are found.

Best regards,

Signature · 1 line
Nicolas