check-uapi: improve portability for testing headers
Subsystems (2)
cover check-uapi: improve portability for testing headers
From: Arnd Bergmann <[email protected]>
While working on a series to clean up some uapi headers, I needed to check the that the actual ABI remains unchanged. I found that scripts/check-uapi.sh works well enough for architectures that have a full toolchain installed, but not with the nolibc compilers I provide on kernel.org.
I ended up with a series addressing three separate issues here, but in the end I can now validate ABI changes across all supported architectures and ABIs.
The third patch depends on a series from Thomas Weißschuh that was just merged into the kbuild-for-next tree, the other ones could also apply to older kernels. I have marked the second patch for backports to stable kernels, this one is what caused me the most work in debugging.
Arnd
Arnd Bergmann (3):
check-uapi: link into shared objects
check-uapi: honor ${CROSS_COMPILE} setting
check-uapi: use dummy libc includes
scripts/check-uapi.sh | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
Signature · 8 lines
2.39.5 Cc: Dodji Seketeli <[email protected]> Cc: John Moon <[email protected]> Cc: Nathan Chancellor <[email protected]> Cc: Nicolas Schier <[email protected]> Cc: Thomas Weißschuh <[email protected]> Cc: [email protected] Cc: [email protected]
PATCH 1/3 check-uapi: link into shared objects
From: Arnd Bergmann <[email protected]>
While testing ABI changes across all architectures, I found that abidiff sometimes produces nonsensical output. Further debugging identified missing or broken libelf support for architecture specific relocations in ET_REL binaries as the source of the problem[1].
Change the script to no longer produce a relocatable object file but instead create a shared library for each header. This makes abidiff work for all of the architectures in upstream linux kernels.
--- scripts/check-uapi.sh | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
scripts/check-uapi.sh +5 -2
@@ -178,8 +178,11 @@ do_compile() { local -r inc_dir="$1" local -r header="$2" local -r out="$3" - printf "int main(void) { return 0; }\n" | \ - "$CC" -c \ + printf "int f(void) { return 0; }\n" | \ + "$CC" \ + -shared \ + -nostdlib \ + -fPIC \ -o "$out" \ -x c \ -O0 \
Signature · 1 line
2.39.5
PATCH 2/3 check-uapi: honor ${CROSS_COMPILE} setting
In reply to Arnd BergmannFrom: Arnd Bergmann <[email protected]>
When ${CROSS_COMPILE} is set, but ${CC} is not set, the logic in check-uapi.sh is different from the top-level Makefile, which defaults to using the cross gcc. This leads to using the native gcc instead of the cross version, resulting in unexpected false-positive and false-negative output.
Use the same logic here that we use in Kbuild for consistency.
--- scripts/check-uapi.sh | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
scripts/check-uapi.sh +6 -5
@@ -33,9 +33,10 @@ Options: -v Verbose operation (print more information about each header being checked). Environmental args: - ABIDIFF Custom path to abidiff binary - CC C compiler (default is "gcc") - ARCH Target architecture for the UAPI check (default is host arch) + ABIDIFF Custom path to abidiff binary + CROSS_COMPILE Toolchain prefix for compiler + CC C compiler (default is "\${CROSS_COMPILE}gcc") + ARCH Target architecture for the UAPI check (default is host arch) Exit codes: $SUCCESS) Success
@@ -198,7 +199,7 @@ do_compile() { run_make_headers_install() { local -r ref="$1" local -r install_dir="$(get_header_tree "$ref")" - make -j "$MAX_THREADS" ARCH="$ARCH" INSTALL_HDR_PATH="$install_dir" \ + make -j "$MAX_THREADS" CROSS_COMPILE="${CROSS_COMPILE}" ARCH="$ARCH" INSTALL_HDR_PATH="$install_dir" \ headers_install > /dev/null }
@@ -407,7 +408,7 @@ min_version_is_satisfied() { # Make sure we have the tools we need and the arguments make sense check_deps() { ABIDIFF="${ABIDIFF:-abidiff}" - CC="${CC:-gcc}" + CC="${CC:-${CROSS_COMPILE}gcc}" ARCH="${ARCH:-$(uname -m)}" if [ "$ARCH" = "x86_64" ]; then ARCH="x86"
Signature · 1 line
2.39.5
PATCH 3/3 check-uapi: use dummy libc includes
In reply to Arnd BergmannFrom: Arnd Bergmann <[email protected]>
Based on Thomas Weißschuh's series to kernel headers to no longer require an installed libc when build testing the uapi headers, the same can now be done for the scripts/check-uapi.sh script.
The only required change here is to add the usr/dummy-include include path.
--- scripts/check-uapi.sh | 1 + 1 file changed, 1 insertion(+)
scripts/check-uapi.sh +1 -0
@@ -191,6 +191,7 @@ do_compile() { -fno-eliminate-unused-debug-types \ -g \ "-I${inc_dir}" \ + "-Iusr/dummy-include" \ -include "$header" \ - }
Signature · 1 line
2.39.5
Re: [PATCH 3/3] check-uapi: use dummy libc includes
Quoted 20 lines · Arnd Bergmann
From: Arnd Bergmann <[email protected]>
Based on Thomas Weißschuh's series to kernel headers to no longer require an installed libc when build testing the uapi headers, the same can now be done for the
scripts/check-uapi.shscript.The only required change here is to add the
usr/dummy-includeinclude path.Signed-off-by Arnd Bergmann <[email protected]>--- scripts/check-uapi.sh | 1 + 1 file changed, 1 insertion(+)scripts/check-uapi.sh +0
-0
scripts/check-uapi.sh do_compile() { · line 194
Quoted 4 lines
scripts/check-uapi.sh +1
-0@@ -191,6 +191,7 @@ do_compile() { -fno-eliminate-unused-debug-types \ -g \ "-I${inc_dir}" \ + "-Iusr/dummy-include" \
What about also using -nostdinc?
scripts/check-uapi.sh do_compile() { · line 197
Quoted 3 lines
scripts/check-uapi.sh +0
-0-include "$header" \ - }
I have a similar (unfinished) patch flying around which also uses usr/dummy-include from the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.
scripts/check-uapi.sh +3 -0
@@ -186,7 +186,9 @@ do_compile() { -std=c90 \ -fno-eliminate-unused-debug-types \ -g \ + -nostdinc \ "-I${inc_dir}" \ + "-I${inc_dir}/../dummy-include" \ -include "$header" \ - }
@@ -197,6 +199,7 @@ run_make_headers_install() { local -r install_dir="$(get_header_tree "$ref")" make -j "$MAX_THREADS" ARCH="$ARCH" INSTALL_HDR_PATH="$install_dir" \ headers_install > /dev/null + rsync -a usr/dummy-include/ "$install_dir"/dummy-include/ } # Install headers for both git refs
Re: [PATCH 3/3] check-uapi: use dummy libc includes
Quoted 6 lines · Thomas Weißschuh
scripts/check-uapi.sh do_compile() {
Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1
-0-g \ "-I${inc_dir}" \ + "-Iusr/dummy-include" \What about also using -nostdinc?
I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.
Quoted 4 lines
I have a similar (unfinished) patch flying around which also uses
usr/dummy-includefrom the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.
Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.
Arnd
Re: [PATCH 3/3] check-uapi: use dummy libc includes
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh
scripts/check-uapi.sh do_compile() {
Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1
-0-g \ "-I${inc_dir}" \ + "-Iusr/dummy-include" \What about also using -nostdinc?
I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.
Ack, your choice.
Quoted 9 lines
Quoted 4 lines
I have a similar (unfinished) patch flying around which also uses
usr/dummy-includefrom the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.
It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.
Thomas
Nicolas Schier
maintainerRe: [PATCH 3/3] check-uapi: use dummy libc includes
Hi Arnd and Thomas,
Quoted 24 lines · Thomas Weißschuh
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh
scripts/check-uapi.sh do_compile() {
Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1
-0-g \ "-I${inc_dir}" \ + "-Iusr/dummy-include" \What about also using -nostdinc?
I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.
Ack, your choice.
Quoted 9 lines
Quoted 4 lines
I have a similar (unfinished) patch flying around which also uses
usr/dummy-includefrom the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.
Quoted 7 lines
It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.
Thomas
Shall I wait some more days for a possible modified patch 3/3? Otherwise I'd like to apply the series to kbuild-next-unstable.
Thanks and kind regards,
Nicolas
Thomas Weißschuh
Reviewed-byRe: [PATCH 3/3] check-uapi: use dummy libc includes
Quoted 25 lines · Nicolas Schier
Quoted 24 lines · Thomas Weißschuh
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh
scripts/check-uapi.sh do_compile() {
Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1
-0-g \ "-I${inc_dir}" \ + "-Iusr/dummy-include" \What about also using -nostdinc?
I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.
Ack, your choice.
Quoted 9 lines
Quoted 4 lines
I have a similar (unfinished) patch flying around which also uses
usr/dummy-includefrom the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.
Quoted 7 lines
Quoted 4 lines
It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.
Shall I wait some more days for a possible modified patch 3/3? Otherwise I'd like to apply the series to kbuild-next-unstable.
It is a somewhat theoretical issue and if it arises at some point we can still fix it up. So this shouldn't block the patches.
For the series:
Thomas
Nicolas Schier
maintainerRe: [PATCH 3/3] check-uapi: use dummy libc includes
Quoted 37 lines · Thomas Weißschuh
Quoted 25 lines · Nicolas Schier
Quoted 24 lines · Thomas Weißschuh
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Thomas Weißschuh
scripts/check-uapi.sh do_compile() {
Quoted 3 lines · Arnd Bergmann
scripts/check-uapi.sh +1
-0-g \ "-I${inc_dir}" \ + "-Iusr/dummy-include" \What about also using -nostdinc?
I just removed it from my version after I found it made no difference, and I wanted to keep the changes shorter. I agree it's slightly cleaner.
Ack, your choice.
Quoted 9 lines
Quoted 4 lines
I have a similar (unfinished) patch flying around which also uses
usr/dummy-includefrom the different kernel versions to avoid mismatches in case something gets removed there. Not sure if it is worth it.Agreed, I certainly don't mind having your version either if anyone cares enough. I suppose it would add a very small build time overhead for the extra copy, but if it does help catch bugs it would be worth the time.
Quoted 7 lines
Quoted 4 lines
It is less about catching bugs, those will already have been caught by the header tests before. But if something is removed from the dummy headers because it became unused, the old UAPI headers won't build anymore against the new dummy-headers.
Shall I wait some more days for a possible modified patch 3/3? Otherwise I'd like to apply the series to kbuild-next-unstable.
It is a somewhat theoretical issue and if it arises at some point we can still fix it up. So this shouldn't block the patches.
Quoted 6 lines
For the series:
Reviewed-by Thomas Weißschuh <[email protected]>Thomas
ok, thanks!
Signature · 1 line
Nicolas
Quoted 18 lines · Arnd Bergmann
From: Arnd Bergmann <[email protected]>
While working on a series to clean up some uapi headers, I needed to check the that the actual ABI remains unchanged. I found that
scripts/check-uapi.shworks well enough for architectures that have a full toolchain installed, but not with the nolibc compilers I provide on kernel.org.I ended up with a series addressing three separate issues here, but in the end I can now validate ABI changes across all supported architectures and ABIs.
The third patch depends on a series from Thomas Weißschuh that was just merged into the kbuild-for-next tree, the other ones could also apply to older kernels. I have marked the second patch for backports to stable kernels, this one is what caused me the most work in debugging.
Quoted 6 lines
Arnd
Arnd Bergmann (3):
check-uapi: link into shared objects
check-uapi: honor ${CROSS_COMPILE} setting
check-uapi: use dummy libc includes
I assume that the shared objects change does not impact catching ABI differences?
Cheers,
Nathan
Nicolas Schier
maintainerQuoted 9 lines · Arnd Bergmann
From: Arnd Bergmann <[email protected]>
While working on a series to clean up some uapi headers, I needed to check the that the actual ABI remains unchanged. I found that
scripts/check-uapi.shworks well enough for architectures that have a full toolchain installed, but not with the nolibc compilers I provide on kernel.org.[...]
Applied to kbuild/linux.git (kbuild-next-unstable), thanks!
[1/3] check-uapi: link into shared objects
https://git.kernel.org/kbuild/c/a261f6df [2/3] check-uapi: honor ${CROSS_COMPILE} setting https://git.kernel.org/kbuild/c/9940ec38 [3/3] check-uapi: use dummy libc includes https://git.kernel.org/kbuild/c/bb25b563
Please look out for regression or issue reports or other follow up comments, as they may result in the patch/series getting dropped, reverted or modified (e.g. trailers). Patches applied to the kbuild-next-unstable branch are accepted pending wider testing in linux-next and any post-commit review; they will generally be moved to the kbuild-next branch in about a week if no issues are found.
Best regards,
Signature · 1 line
Nicolas